Open in app

Sign in

Write

Sign in

Using Python to check EIDR Token Status

Hitesh Pau
8 min readSep 23, 2020

Before you start reading this article, please read my previous article Using Python to access the EIDR registry with EIDR REST APIs.

Pre-requisites

  1. You have Python installed. For this tutorial, we are using Python version 3.7.4. If not, you can download Python from https://www.python.org/ Please refer to Python documentation to install Python on your machine.
  2. You have a Text editor installed on your machine. There are many python friendly editor, which helps writing Python code easier.

For Mac, you can use SublimeText, Atom, TextWrangler

For Windows, you can use Notepad, Notepad++, SublimeText, Atom

3. You can also install Jupyter, which is a web-based interactive Python development environment. You can download from https://jupyter.org/

Assumption

  1. You know basic Python programming.
  2. You are a member of EIDR and have credentials to access EIDR Registry

EIDR Rest APIs

The EIDR system provides various services using a REST based interface in combination with HTTP 1.1 (see RFC 2616).

Here is the link to the EIDR Rest API document

Note: Public services do not necessarily mean open access. Ingesting or registering data into EIDR is controlled, while reading data from EIDR is generally not restricted.

EIDR Status Lookup Service API

Refer to 2.3.2 Status lookup service (page 18) in the EIDR Rest API document for the request call and parameters.

Let’s get started…

Refer to Using Python to access the EIDR registry with EIDR REST APIs to do the following -

  1. Install the required python packages
  2. Authenticate and authorize to access EIDR Rest APIs
import requests, base64, hashlib
from bs4 import BeautifulSoup
import pandas as pd
UserID = '10.5238/xxxxxxxx'. # enter your EIDR User Id
Pwd = '************' # enter your EIDR password
PartyID = '10.5237/xxxxxxxxx' # enter your EIDR party ID
url = 'https://sandbox1.eidr.org:443/EIDR/' # EIDR Registry URL#Encrypt the credentials
PasswordShadow = base64.b64encode(hashlib.md5(pwd.encode('utf-8')).digest()).decode('utf8')auth_str = '%s:%s:%s' % (UserID, PartyID, PasswordShadow)headers = {'Authorization' : 'Eidr {}'.format(auth_str), 'Accept': 'text/xml', 'Content-Type': 'text/xml'}

Now we write a function to get the status of the EIDR token. Refer to 2.3.4 Status lookup service (page 27) in the EIDR Rest API document for the request call and parameters.

def getTokenStatus(token):
    req = url + 'status/token/' + token
    resp = requests.get(req, headers=headers)
    return resp.content

Let’s test the function by passing a token and seeing the result (or response)

#get the token status
token_status_response = getTokenStatus('1597356462419640618')#display the token status response xml
soup = BeautifulSoup(token_status_response, 'xml')
print(soup.prettify())

Here is the response XML

<?xml version="1.0" encoding="utf-8"?>
<Response version="2.6.0" xmlns="http://www.eidr.org/schema">
 <Status>
  <Code>
   0
  </Code>
  <Type>
   success
  </Type>
 </Status>
 <RequestStatus>
  <Token>
   1597356462419640618
  </Token>
 </RequestStatus>
 <RequestStatusResults>
  <CurrentSize>
   1
  </CurrentSize>
  <TotalMatches>
   1
  </TotalMatches>
  <OperationStatus>
   <Token>
    1597356462419640618
   </Token>
   <Status>
    <Code>
     0
    </Code>
    <Type>
     success
    </Type>
   </Status>
   <ID>
    10.5240/7331-690C-D843-BE10-3CF2-X
   </ID>
  </OperationStatus>
 </RequestStatusResults>
</Response>

The response XML shows that the token is resolved successfully and an EIDR ID (10.5240/7331–690C-D843-BE10–3CF2-X) is generated for that token. We can extract the EIDR Id using the BeautifulSoup package.

#get the token status
token_status = soup.Response.Status.Type.contents[0]#get the operation status, which could be different from the main token status. Examples are duplicate, validation error, bad id...
token_operation_status = soup.Response.RequestStatusResults.OperationStatus.Type.contents[0]#get the EIDR Id, if the token is resolved
token_status_ID = soup.Response.RequestStatusResults.OperationStatus.ID.contents[0]#display the status
print('Status: {}    OperationStatus: {}   EIDR ID: {}'.format(token_status, token_operation_status, token_status_ID))

The above token resolved into an EIDR ID, which is great, but that is not always the case. There are other scenarios, where we see different results.

Duplicate

Here we passed a token which resulted in a Duplicate status

#get the token status
token_status_response = getTokenStatus('1591648141088364253')#display the token status response xml
soup = BeautifulSoup(token_status_response, 'xml')
print(soup.prettify())

Here is the response XML

<?xml version="1.0" encoding="utf-8"?>
<Response version="2.6.0" xmlns="http://www.eidr.org/schema">
 <Status>
  <Code>
   0
  </Code>
  <Type>
   success
  </Type>
 </Status>
 <RequestStatus>
  <Token>
   1591648141088364253
  </Token>
 </RequestStatus>
 <RequestStatusResults>
  <CurrentSize>
   1
  </CurrentSize>
  <TotalMatches>
   1
  </TotalMatches>
  <OperationStatus>
   <Token>
    1591648141088364253
   </Token>
   <Status>
    <Code>
     1
    </Code>
    <Type>
     duplicate
    </Type>
   </Status>
   <Duplicate>
    <ID>
     10.5240/1209-DB55-259A-32DB-568A-B
    </ID>
   </Duplicate>
   <Duplicate>
    <ID>
     10.5240/E30E-7657-63B8-3CAB-5060-R
    </ID>
   </Duplicate>
   <Duplicate>
    <ID>
     10.5240/0EF7-8253-4365-2869-0327-3
    </ID>
   </Duplicate>
   <Duplicate>
    <ID>
     10.5240/B3B8-FE2D-8F36-6C50-AD96-N
    </ID>
   </Duplicate>
   <Duplicate>
    <ID>
     10.5240/124E-38C5-038A-656A-CCDC-V
    </ID>
   </Duplicate>
  </OperationStatus>
 </RequestStatusResults>
</Response>

In the above response, the main token status is successful, but the operation status type is duplicate. Here we have multiple duplicate IDs for this token. Typically for a content analyst this would require further investigation on the token. We can extract the duplicate IDs using the BeautifulSoup package.

#create an empty list to store the tokens
token_status_Ids = []#check if the operation status is duplicate
if token_operation_status == 'duplicate':
    #loop thru each duplicate ID and add it to the list
    for id in soup.Response.RequestStatusResults.OperationStatus.findAll('Duplicate'):
        token_status_Ids.append(id.get_text('ID'))
        
    #display the duplicate Id list
    print(token_status_Ids)

Validation Error

Here we passed a token which resulted in a validation error

#get the token status
token_status_response = getTokenStatus('1592869486580440852')#display the token status response xml
soup = BeautifulSoup(token_status_response, 'xml')
print(soup.prettify())

Here is the response XML

<?xml version="1.0" encoding="utf-8"?>
<Response version="2.6.0" xmlns="http://www.eidr.org/schema">
 <Status>
  <Code>
   0
  </Code>
  <Type>
   success
  </Type>
 </Status>
 <RequestStatus>
  <Token>
   1592869486580440852
  </Token>
 </RequestStatus>
 <RequestStatusResults>
  <CurrentSize>
   1
  </CurrentSize>
  <TotalMatches>
   1
  </TotalMatches>
  <OperationStatus>
   <Token>
    1592869486580440852
   </Token>
   <Status>
    <Code>
     4
    </Code>
    <Type>
     validation error
    </Type>
    <Details>
     Another episode exists with the same Parent and 'HouseSequence' or 'DistributionNumber': 10.5240/59F5-64D4-E54C-D579-1F4A-Y
    </Details>
   </Status>
  </OperationStatus>
 </RequestStatusResults>
</Response>

In the above response, the main token status is successful, but the operation status type is validation error. To extract this information, we can use the below code

token_status = soup.Response.Status.Type.contents[0]
token_operation_status = soup.Response.RequestStatusResults.OperationStatus.Type.contents[0]
token_operation_status_details = soup.Response.RequestStatusResults.OperationStatus.Status.Details.contents[0]print('Status: {}    OperationStatus: {}   Details: {}'.format(token_status, token_operation_status, token_operation_status_details))

Other Errors

There are other errors as well. Here we get the bad token error

#get the token status
token_status_response = getTokenStatus('159286948658044085')#display the token status response xml
soup = BeautifulSoup(token_status_response, 'xml')
print(soup.prettify())

Here is the response XML

<?xml version="1.0" encoding="utf-8"?>
<Response version="2.6.0" xmlns="http://www.eidr.org/schema">
 <Status>
  <Code>
   6
  </Code>
  <Type>
   bad token error
  </Type>
 </Status>
</Response>

In the above response, the main token status type is bad token error. You can extract it as below

token_status = soup.Response.Status.Type.contents[0]
print('Status: {} '.format(token_status))

For more details on other status codes and operation codes, please refer to 2.1.4 Codes and Descriptions (page 9) in the EIDR Rest API document.

Considering the above token responses, we need to conditionally extract token status. To do so, let’s write another function to extract the token status descriptions.

def getTokenStatusDesc(tokenStatusXML):
    
    #initialize a variable to hold the status description
    tokenStatus = ''
    
    #read the token status response xml
    soup = BeautifulSoup(tokenStatusXML, 'xml')
    
    #get the token status code and convert into an integer
    token_status_code = int(soup.Response.Status.Code.contents[0])#check the token status
    if token_status_code == 0:
        
        #get the token operation status code and convert to integer
        token_operation_status_code = int(soup.Response.RequestStatusResults.OperationStatus.Code.contents[0])#check the token operation status code
        if token_operation_status_code == 0:
            resolved_eidr = soup.Response.RequestStatusResults.OperationStatus.ID.contents[0]
            tokenStatus = resolved_eidr
            
        elif token_operation_status_code == 1: #duplicate
            #list to store the duplicate EIDRs found
            token_status_Ids = []#loop thru each duplicate EIDR Id and save them into a list
            for id in soup.Response.RequestStatusResults.OperationStatus.findAll('Duplicate'):
                token_status_Ids.append(id.get_text('ID'))#covert the list into a comma separated string and concatenate with the status message
            tokenStatus = 'Possible Duplicates: ' + ', '.join(token_status_Ids)
            
        elif token_operation_status_code == 4: #validation error
            val_error = soup.Response.RequestStatusResults.OperationStatus.Status.Details.contents[0]
            tokenStatus = val_error
            
        else:
            #print('Token resolution error')
            tokenStatus = 'Token resolution error'
    else:
        token_status = soup.Response.Status.Type.contents[0]
        tokenStatus = token_status
        
    return tokenStatus

The above function captures all the different scenarios discussed above. If you encounter any other status, you can modify the above function to accommodate it.

Let’s test the function to see if it returns the right status description

resp = getTokenStatus('1592869486580440852')
getTokenStatusDesc(resp)

It returns the correct description. The above code test’s only one token. A real use case would be to extract status for multiple tokens.

Let’s write code to read a text file with multiple token and get the status for each and export the output into an excel file.

Create a text file in a text editor and paste the below tokens or add your own tokens.Save the file as input_tokens.txt

1597356462419640618
1597696316125748968
1597696762145749158
1597697995996749652
1597697996415749654
1597697996840749656
1597698762875749902
1597698763326749904
1597698763763749905
1597698764201749908
1597698764638749912
1592869486580440852
1591648141088364253
159286948658044085

Below code does the following -

  1. Set’s a variable with the filename. In our case, it is input_tokens.txt.
  2. Open the file and split the tokens in a list of tokens
  3. Loop thru each token and call the functions that we created i.e. getTokenStatus and getTokenStatusDesc
  4. Save the results in a list
  5. Create a new pandas dataframe using the results list.
#set the inputfile variable with the token filename
inputFile = 'input_tokens.txt'#open the tokenfile and split each token into a list object
with open(inputFile) as f:
    tokens = f.read().splitlines()#create an empty list to store the token status
output_list = []#loop thru each token to get the status
for token in tokens:
    resp = getTokenStatus(token)
    status = getTokenStatusDesc(resp)
    
    print('Token: {}  Status: {}'.format(token, status))
    output_list.append([token, status])df_output = pd.DataFrame(output_list, columns =['Token', 'Status'])
df_output.head()

Here is the output of the df_output.head()

Once the dataframe is created, you can export it to excel

df_output.to_excel('token_status.xlsx')

Here is the complete python code

import requests, base64, hashlib
from bs4 import BeautifulSoup
import pandas as pdUserID = '10.5238/xxxxxxxx'. # enter your EIDR User Id
Pwd = '************' # enter your EIDR password
PartyID = '10.5237/xxxxxxxxx' # enter your EIDR party ID
url = 'https://sandbox1.eidr.org:443/EIDR/' # EIDR Registry URL#Encrypt the credentials
PasswordShadow = base64.b64encode(hashlib.md5(pwd.encode('utf-8')).digest()).decode('utf8')auth_str = '%s:%s:%s' % (UserID, PartyID, PasswordShadow)headers = {'Authorization' : 'Eidr {}'.format(auth_str), 'Accept': 'text/xml', 'Content-Type': 'text/xml'}def getTokenStatus(token):
    
    req = url + 'status/token/' + token
    resp = requests.get(req, headers=headers)
    return resp.contentdef getTokenStatusDesc(tokenStatusXML):
    
    #initialize a variable to hold the status description
    tokenStatus = ''
    
    #read the token status response xml
    soup = BeautifulSoup(tokenStatusXML, 'xml')
    
    #get the token status code and convert into an integer
    token_status_code = int(soup.Response.Status.Code.contents[0])    #check the token status
    if token_status_code == 0:
        
        #get the token operation status code and convert to integer
        token_operation_status_code = int(soup.Response.RequestStatusResults.OperationStatus.Code.contents[0])        #check the token operation status code
        if token_operation_status_code == 0:
            resolved_eidr = soup.Response.RequestStatusResults.OperationStatus.ID.contents[0]
            tokenStatus = resolved_eidr
            
        elif token_operation_status_code == 1: #duplicate
            #list to store the duplicate EIDRs found
            token_status_Ids = []            #loop thru each duplicate EIDR Id and save them into a list
            for id in soup.Response.RequestStatusResults.OperationStatus.findAll('Duplicate'):
                token_status_Ids.append(id.get_text('ID'))            #covert the list into a comma separated string and concatenate with the status message
            tokenStatus = 'Possible Duplicates: ' + ', '.join(token_status_Ids)
            
        elif token_operation_status_code == 4: #validation error
            val_error = soup.Response.RequestStatusResults.OperationStatus.Status.Details.contents[0]
            tokenStatus = val_error
            
        else:
            #print('Token resolution error')
            tokenStatus = 'Token resolution error'
    else:
        token_status = soup.Response.Status.Type.contents[0]
        tokenStatus = token_status
        
    return tokenStatus#set the inputfile variable with the token filename
inputFile = 'input_tokens.txt'#open the tokenfile and split each token into a list object
with open(inputFile) as f:
    tokens = f.read().splitlines()#create an empty list to store the token status
output_list = []#loop thru each token to get the status
for token in tokens:
    resp = getTokenStatus(token)
    status = getTokenStatusDesc(resp)
    
    print('Token: {}  Status: {}'.format(token, status))
    output_list.append([token, status])df_output = pd.DataFrame(output_list, columns =['Token', 'Status'])#export the data to an excel file
df_output.to_excel('token_status.xlsx')

This completes the tutorial on how to get EIDR token status for multiple tokens and export to excel.

Also check out:

how to get EIDR data from Alternate Ids

Eidr
Eidr Api
Python
Tokens
Eidr Tokens

--

--

Hitesh Pau

Written by Hitesh Pau

10 Followers

Data Management and Operations Strategy

Help

Status

About

Careers

Blog

Privacy

Terms

Text to speech

Teams