2.14.2. IDP Recognition

Note:

The IDP Recognition feature is only available with the installation of the additional package dfx-idp.

Document recognition using IDP technology is an existing feature of the Nectainium platform. The functionality works without changes; however, the shortcuts used for configuring IDP recognition have been renamed and relocated.

2.14.2.1. Configure IDP Recognition

Configuring IDP recognition consists of the following steps:

1. Select the Azure or Google provider and create an account to obtain a key:

   • Create an Azure account
   • Create a Google Cloud account

2. Create an IDP entity — define what data needs to be recognized in documents and how it should be structured. Alternatively, you can use the default entities already available in the system:

   a. Prebuilt-invoice (for the Azure provider)

   b. Prebuilt-receipt (for the Azure provider)

   c. INVOICE_PROCESSOR (for the Google provider)

3. Add a model — connect the created Azure or Google profile to the Nectainium platform using the key.

4. Create a recognition template — configure how documents will be recognized and where the recognized data will be stored.

2.14.2.1.1. Create a Provider Account

Azure

Instructions for creating an Azure subscription for using the IDP feature in the Nectainium system with the Azure Form Recognizer provider.

1. Create a new Azure account or sign in if you already have one.

2. Go to the Azure Portal home page.

3. Select Create a resource.

   

4. Select the resource type Document Intelligence (form recognizer), then select Create.

   

5. In the new window, select Start.

   

6. In the new window, select Try Azure for free.

   

7. In the new window, agree to the customer agreement and select Next.

   

8. Enter your card details.

   

9. In the next window, confirm the security check by selecting Next.

   

10. In the new window, sign in to your account using the Sign In button.

    

11. In the next window, select Create a resource.

    

12. Find Document Intelligence (form recognizer) and select Create.

    

13. Fill in the required fields, select Price Tier = Free F0 for a free trial. Select Next.

    

    Example:

    

14. If needed, configure the network access settings for the resource. Select Next.

    

15. Select Next.

    

16. If needed, configure the resource tags. Select Next.

    

17. Select Create.

    

18. Wait for the resource to deploy. When complete, select Go to resource.

    

19. In the Keys and Endpoint section, copy the KEY1 and Endpoint values.

    

20. In the system, create an IDP profile for the AZURE provider using the KEY1 and Endpoint values. Also add the default IDP entities included with the system: Invoice for invoice recognition and Receipt for receipt recognition. To create the profile, log in to the system with Developer permissions and go to Administration — Settings — IDP Provider Profiles.

Google Cloud

Instructions for creating a Google Cloud subscription for using the IDP feature in the Nectainium system with the Google Form Recognizer provider.

Sign in to Google Cloud with an existing Google account (or create a new one if you don't have one).

1. Select Create Project.

   

2. Enter a project name (or leave the auto-generated name) and select Create.

   

3. Next, you need to activate billing — create a trial subscription or select a paid one. Open the menu and select Billing.

   

4. Select MANAGE BILLING ACCOUNT, then ADD BILLING ACCOUNT.

   

5. Enter company information and bank card details. Start the free trial.

   

6. Next, enable access to the Document AI API. Select Next, then Enable.

   

7. Next, create a service account and obtain API keys. Go to create a Service Account in Google Cloud Console. Enter the account information, select CREATE AND COMPLETE, then CONTINUE, then DONE.

   

8. Now create the keys. Open the Actions menu, then select Manage keys.

   

   Select ADD KEY, then Create new key.

   

   Select Key type = JSON and select CREATE. Save the key to your computer.

   

9. Now create a processor. In the Google Cloud menu, search for Document AI and select it.

   

   Select EXPLORE PROCESSORS.

   

   Select Invoice Parser and select CREATE PROCESSOR.

   

   Enter the processor name INVOICE_PROCESSOR and select CREATE.

   

   As a result, you will receive the processor information, which you will need when creating the IDP profile in the system.

   

10. Log in to the system with Developer permissions and go to Administration — Settings — IDP Provider Profiles. Select Add profile.

    Select Provider = Google Form Recognizer, enter a name, and paste the credentials in JSON format (the keys saved to your computer). Select + Add to add a model.

    Select the IDP entity available in the system = INVOICE_PROCESSOR Invoice, enter the model ID matching the processor name in Google Cloud, and the URL address — copy it from the Prediction endpoint in Google Cloud, but not in full, only starting from the word projects. Select Add.

Select Add again to create the IDP Google provider profile.

2.14.2.1.2. Create an IDP Entity

Note:

The IDP Entities feature is only available with the installation of the additional package dfx-idp. This feature is used for intelligent scanning of paper documents and their subsequent conversion to electronic format.

An IDP entity is a set of document recognition attributes provided by the recognition provider (Azure or Google). The entity attributes are then used to create form recognition templates. Using these templates, the system maps (links) the recognized data to the corresponding document type attributes.

1. In the navigation panel, select the Studio 1 workspace.

2. Select the AI Center 2 shortcut group, then select the AI Document Recognition 3 shortcut group.

3. Select the IDP Entities 4 shortcut.

4. In the toolbar, select Create 5.

   

5. Fill in the fields using the guidance in the table below.

   

Field	Description
Code*	A unique identifier for the entity. The code must match the model code deployed with the Azure or Google provider. The code must be unique. The code must be short (typically up to 10 characters). Use only Latin letters and digits. Example: passport_recognition
Name*	Enter the desired name for the IDP entity. Example: Passport Data Recognition ℹ️ Note: You can set the entity name for different languages. To do so, in the Name field, select the icon and fill in the fields for other languages. Then select the Apply button.
Description	A brief description of the entity's purpose. Example: Entity for passport data recognition
Provider*	Select one of the following form recognition providers: Azure Form Recognizer — for advanced recognition using Azure AI Document Intelligence. Google Form Recognizer — for advanced recognition using Google Cloud Vision API.

Note:

Fields marked with * are required.

6. In the toolbar, select Save 1.

7. In the IDP Entity Attributes table, select Create 2.

   

8. Fill in the fields using the guidance in the table below.

   

Field	Description
Code*	A unique identifier for the IDP entity attribute — must match the code provided by the provider.
Name*	Enter the desired name for the IDP entity attribute. ℹ️ Note: You can set the attribute name for different languages. To do so, in the Name field, select the icon and fill in the fields for other languages. Then select the Apply button.
Description	A description of the IDP entity attribute.
Data Type*	Select the data type for the attribute: String Date Number Address Currency Array ℹ️ Note: If you selected the Array data type, you will need to create additional array element attributes. To do so, select Save 1, then in the Array Element Attributes field, select the Create 2 icon.

Note:

Fields marked with * are required.

9. In the toolbar, select Save.

2.14.2.1.3. Add a Model

1. In the navigation panel, select the Studio 1 workspace.

2. Select the AI Center 2 shortcut group, then select the AI Models 3 shortcut.

   

3. In the toolbar, select Add a model 1.

4. In the dropdown list, select IDP Recognition 2.

   

5. Fill in the fields using the guidance in the table below.

   

Field	Description
Provider*	Select one of the available options: Azure Form Recognizer Google Document Recognition
Name*	Enter a short and descriptive name.
Description	Add additional information about the configuration.

---

For Azure

Field	Description
Key*	Enter your API key obtained from Azure when registering the provider account. To find your key, follow these steps: Sign in to your Azure account. In the settings, go to the Keys and Endpoint section. Find the KEY 1 field and copy its value.
URL Address*	Enter the endpoint URL for accessing the recognition service, obtained from Azure when registering the provider account. To find your URL, follow these steps: Sign in to your Azure account. In the settings, go to the Keys and Endpoint section. Find the Endpoint field and copy its value.
Models*	You can use the already added prebuilt models or add your own. To add your own, select the Add button. In the IDP Entity field, select from the list the IDP entity you created in the previous step, or select one of the default ones: Prebuilt-invoice Prebuilt-receipt In the Model ID field, enter the corresponding identifier. Select the Add button.

---

For Google

Field	Description
Credentials*	Upload the JSON file with the service account credentials for Google, which you downloaded when creating the Google Cloud account.
Models*	In the IDP Entity field, select from the list the IDP entity you created in the previous step, or select the default entity INVOICE_PROCESSOR. In the Model ID field, enter the same name as in the Name field of your processor in Google Cloud. In the URL Address field, enter the URL address — copy it from the Prediction endpoint field in Google Cloud, but not in full, only starting from the word projects. Select the Add button.

Note:

Fields marked with * are required.

6. After filling in all the fields, select the Add button.

2.14.2.1.4. Create a Recognition Template

1. In the navigation panel, select the Studio 1 workspace.

2. Select the AI Center 2 shortcut group, then select the AI Document Recognition 3 shortcut group.

3. Select the Form Recognition Templates 4 shortcut.

4. In the toolbar, select + 5.

   

5. Fill in the fields 1 using the guidance in the table below, then in the toolbar select Save 2.

   

Field	Description
Code*	A unique identifier for the template. The code must be unique. The code must be short (typically up to 10 characters). Use only Latin letters and digits.
Name*	Enter a clear name for the template that will be displayed in the system interface. For example: "Invoice Recognition" or "Passport Template". ℹ️ Note: You can set the template name for different languages. To do so, in the Name field, select the icon and fill in the fields for other languages. Then select the Apply button.
Description	Add a brief description of the template's purpose and usage specifics.
Recognition Method*	Select the Form Recognition Provider option.
Provider*	Select one of the available recognition providers: Tesseract Recognition — created as a universal template, not tied to a specific document type. The template configures the mapping between a region on the document from which text will be read and the document attribute where the text will be stored. This recognition method can only be used with a scanner. First, the scanner must be connected to the platform, after which the scanned file in JPEG or TIFF format is passed to the Tesseract module. Tesseract adds a text layer to the uploaded file, which is used for recognition. The text from the document fragment defined by coordinates in the template is then written as a text value to the corresponding document attribute. The template can be used for attributes of the following types: Document Image File Azure Form Recognizer — for advanced recognition using Azure AI Document Intelligence. The template is created tied to a specific document type. Google Form Recognizer — for advanced recognition using Google Cloud Vision API. The template is created tied to a specific document type.

---

Settings for the "Tesseract" Provider

Template — Enter the template settings as a JSON array, where each object contains the following parameters:

[
  {
    "attribute": "",                    // The attribute code where you want to write the extracted results
    "top": null,                        // The X coordinate of the top-left corner of the recognition area
    "left": null,                       // The Y coordinate of the top-left corner of the recognition area
    "width": null,                      // The width of the recognition area
    "height": null,                     // The height of the recognition area
  }
]

Note:

If you specify an empty JSON array [] in the Template field, the system will only perform text recognition on the scanned file and add a text layer to it. In this case, the extracted text values will not be written to any document attribute.

For example, to recognize the document date from the first page, in an area of 200x40 pixels located 100 pixels from the top and 50 pixels from the left:

[
  {
    "attribute": "documentDate",
    "top": 100,
    "left": 50,
    "width": 200,
    "height": 40,
  }
]

---

Settings for the "Google" and "Azure" Providers

Field	Description
Entity*	Select the required IDP entity from the list.
Document Type*	Select the document type for which the recognition template is being created. This determines which attributes will be available for mapping.
"Add Mapping" Button	Select the Add Mapping button to create a link between the data to be recognized in the document and the location where this data will be stored in the system. You can create multiple such links within one template.
Recognition Result Layer	Select one of the following options: Entity — used for working with a pre-trained recognition model (Azure, Google). The system already "knows" where to look for specific data in a document of a given type. No keywords or patterns are required — the model determines the required fields on its own. Recognition fields are selected from the model's standard list (for example, "Due Date", "Remittance Address"). Key-Value Pairs — used for searching data by specific markers in the document. Requires more manual configuration but provides more control over the recognition process.

---

Settings for the "Entity" Result Layer Type

Field	Description
Recognition Result Field*	Select the required model field from the list that contains the recognized text. For example: "Due Date" for the payment date. Below the field you will see a hint in English explaining the purpose of this field.
Recognition Result Subfield	This field appears when the field selected above is complex and contains subfields. For example: addresses consisting of street, city, and region. Select the required subfield from the list.
Document Attribute Type*	Select one of the following attribute types to configure mapping for: Library Attribute — custom attributes Document Attribute — standard attributes
Document Attribute*	Select the specific attribute from the list where the recognized value will be stored.

---

Settings for the "Key-Value Pairs" Result Layer Type

Field	Description
Token Key*	Enter the keyword by which the required text will be found.
Token Regular Expression*	Enter a regular expression — a pattern used to define the structure of the text to be recognized. Examples: \d+ — to search for any sequence of digits (for example, a number) \d{2}\.\d{2}\.\d{4} — to search for a date in DD.MM.YYYY format [A-Z]{2}\d{6} — to search for a code consisting of 2 letters and 6 digits See Regular Expression Syntax.
Document Attribute Type*	Select one of the following attribute types to configure mapping for: "Library Attribute" — custom attributes "Document Attribute" — standard attributes
Document Attribute*	Select the specific attribute from the list where the recognized value will be stored.

Notes:

• Fields marked with * are required.

• For table recognition, the array mapping feature is supported. To use it, a pre-trained model (pre-built model) must be used.

2.14.2.2. Apply the Tesseract Recognition Template

Since the Tesseract form recognition template is created as a universal template and is not tied to a specific document type, you can apply it to any document types. To do so, follow these steps:

1. In the navigation panel, select the Studio 1 workspace.

2. Select the Document Types 2 shortcut.

3. Open the document type to which you want to apply the template 3.

   

4. Go to the Constructor tab.

   

5. Add a Document Image or File 1 attribute to the form, into which the scanned and recognized file will be uploaded.

6. Add Text 2 attributes to the form, into which the text fragments defined by coordinates will be written according to the recognition template settings.

   

7. Hover over the file attribute, then select the icon to configure actions.

   

8. Select the checkboxes next to the following actions 1:

   • Scan + OCR
   • Scan
   • Scan Settings

9. Select Save 2.

   

10. Select the file attribute 1, then in the attribute Settings menu find the Form Recognition Templates 2 field and select the required template from the list.

    

11. In the toolbar, select Save.

    

12. Create a document of this type.

    1. In the navigation panel, select the Documents 1 workspace.

    2. Select the Documents 2 shortcut, then in the toolbar select + 3.

       

    3. In the Document Type 1 field, select the required type, then select Create 2.

       

13. After you create the document, its form will open. In the file attribute (which you configured in the previous steps), select the icon to configure the scanner.

    

14. In the Scanner 1 section, select the connected scanner from the list. If you have not yet connected a scanner, see Scanner Settings.

15. In the File Format 2 section, select JPEG or TIFF. Other formats are not supported for recognition using Tesseract.

16. Select Change 3 to save the changes.

    

17. In the file attribute, select the icon to scan the document and recognize it.

    

18. After scanning is complete, the file attribute will contain the file with a text layer (you will be able to select text in the file and use it anywhere as needed). The required text fragments will also be written to the attribute values according to the recognition template settings.

2.14.2.3. Apply an Azure or Google Recognition Template

Note:

Before applying the template, make sure you have configured IDP recognition.

There are two ways to apply the Azure or Google provider recognition template:

• Method 1: via a system shortcut.
  In this method, recognition can only be performed by a platform administrator. The recognition instructions are identical to the AI recognition instructions — Method 1: via a system shortcut.

• Method 2: via a custom shortcut.
  In this method, recognition can be performed by any platform users to whom the administrator grants access to the shortcut. The recognition instructions are identical to the AI recognition instructions — Method 2: via a custom shortcut.

Note:

Unlike AI recognition, IDP recognition supports processing only one document at a time.

---

After you recognize a document using IDP, you can highlight the content you need in the scanned document to quickly find the required information. To do so, select the attribute whose content you want to see in the document 1, and the required fragment of the document will be highlighted in green 2.