Skip to main content

Search

Table of сontents

Introduction​

InCountry Border supports the unredaction of search requests that deal with regulated and sensitive data. Such search requests are performed against two data sources:

  1. InCountry platform that stores regulated data

  2. application backend that queries non-regulated data from its database

This documentation will explain how you can integrate search capabilities into your web application and what you need to implement on your side, as well as technical limitations that may arise.

Solution overview​

This solution allows users to search data by regulated fields stored in InCountry Vault, as well as by non-regulated fields stored in the application database. There are three possible situations for search:

  1. Search by non-regulated fields: Border simply forwards the search request to the application backend. The response from the application backend may contain redacted values (tokens/hashes), which are unredacted by Border using the regular unredaction mechanism.

  2. Search by regulated fields: Border first looks up appropriate records in InCountry Vault and then passes the list of found records' identifiers to the application backend. It may then process the found records' identifiers in one of the following ways:

    • Verify the user’s access to records against the identifiers provided by Border: The application backend must have ACL (access-control list) that can return HTTP 200/422 response. In this case, the response from Border to the customer frontend will have a predefined format.

    • Return a filtered list of records: This can also be done with the application backend using ACL. In this case, the response from the application backend may contain redacted values (tokens/hashes), they are further unredacted by Border using the regular unredaction mechanism.

  3. Search by both regulated and non-regulated fields: Border first searches for appropriate records in InCountry Vault by regulated fields and then sends the list of found records' identifiers combined with the search query by non-regulated fields to the application backend. The application backend should then return a filtered list of records (after checking ACL) and the provided search query. In this case, the response from the application backend will contain redacted values (hashes/tokens), they are further unredacted by Border using the regular unredaction mechanism.

Application backend limitations​

The application backend must meet the following requirements:

  • It must have a RESTful API that returns and accepts data in the JSON format only.

  • It must have token-based authentication (OAuth, JWT).

  • It must operate with entities/resources and return a list of entities/resources having some kind of ID as a response to a search query.

  • It must have an endpoint for filtering records by the list of IDs or for approving the list of records that will be sent to the user.

  • It must have an endpoint for authentication check.

Authentication and authorization in application backend​

Border does not have a mechanism to check the authentication of the incoming request, so the application backend is responsible for the authentication check. Additionally, Border does not have an authorization mechanism for verifying user’s access to a specific record, so the application backend is responsible for the authorization check as well.

Authentication mechanism. Basic authentication using a pre-request​

Flow diagram​

The following is an explanation of the flow diagram:

  1. The client (browser) sends a search request to Border.

  2. Border verifies that the client is identified by the application backend.

  3. Border sends the client's authentication header to the backend.

  4. The application backend responds whether it identifies the client or not.

    1. If the application backend does not identify the client, it responds to the client with an access error.

    2. If the backend identifies the client, Border looks up records matching the provides criteria in InCountry Vault. Border transforms the request according to the configuration.

  5. After querying the records, Border extracts the record identifiers and passes them to the application backend (a special endpoint is allocated for this).

  6. The application backend returns redacted records that match the provided filter. These records are filtered out according to the list with record identifiers (ACL applied).

  7. Border unredacts redacted values for provided records (that has undergone filtration by identifier).

note

  • The application backend should return a non-2xx (>=400) HTTP response code in case of an unauthorized request.

  • Border should not be able to access any records without being authenticated by the application backend.

  • The application backend should only return records that the client is authorized to access.

Record lookup with record ids​

You can use this approach to search for record by regulated and non-regulated fields.

The application backend should have an endpoint that can take a list of record identifiers from Border with additional optional filters and return the list of records (identifiers) which the user (with the provided Auth header) has access to.

Please examine the following example:

The user wants to find all customers with name “John” (regulated field) and a favorite car “Ford” (non-regulated field).

1. Request from client to Border:

POST https://sa-proxy-mt-01.incountry.io/x-inc-integration_id/customers/search
{
  name: "John",
  car: "Ford"
}

2. Border sends a dummy auth request to application backend:

POST https://backend.client.com/dummy-auth
Authorization: Bearer TOKEN_FROM_INITIAL_REQUEST
=> HTTP 200 OK

3. Border looks up all records in InCountry where name is "John" and
gets the list of record identifiers: 'id1', 'id2', 'id3'

4. Border sends a request to application backend with parameters:
{
  ids: ['id1', 'id2', 'id3']
  car: 'Ford'
}

GET https://backend.client.com/users?id=id1&id=id2&id=id3&car=Ford
Authorization: Bearer TOKEN_FROM_INITIAL_REQUEST
=> HTTP 200 OK
{
  users: [
    {
      id: 'id1',
      name: 'hashed-value-1',
      address: 'hashed-value-addr-1',
      car: 'Ford'
    }, {
      id: 'id3',
      name: 'hashed-value-3',
      address: 'hashed-value-addr-3',
      car: 'Ford'
    }
  ]
}

5. Border transforms the response from application backend, replacing hashed 
values with clear text and returns the response to Customer
=> HTTP 200 OK
{
  users: [
    {
      id: 'id1',
      name: 'John Smith',
      address: '32 Rustaveli st.',
      car: 'Ford'
    },
    {
      id: 'id3',
      name: 'John Doe',
      address: '10 Mukuzani lane',
      car: 'Ford'
    }
  ]
}

Find filter (criteria) modifications in request example​

Border endpoint (partial configuration):

"criteriaMapping": {
  "map": {
    "$.name": "key10",
    "$.favoriteDish": "search_keys"
  }
}

Initial request from application backend to Border:

{
  name: "John", // regulated field, present in config.redactions[].criteriaMapping.map
  car: "Ford" // non-regulated field, not specified in config
}

Search request in InCountry Vault:

{
  "key10" : "John"
}

Search request to the application backend (initial search request parameters without all regulated fields and with ids field added)

{
  ids: ['id1', 'id2', 'id3']
  car: 'Ford'
}
note

If InCountry Vault does not return any record identifiers for the provided filter against regulated fields (as no records matching filter criteria are found), this means that the request does not handle any regulated values, so there is no need to send a request with filtered records to Border.

Response modification using normal unredaction rule example​

Example of the respons from searchEndpoint:

{
  users: [
    {
      id: 'user-0.6213436337059335',
      name: 'bLXwbGZwU6joGzuxG6kItYeqFJN0q5Z5',
      car: 'Ford',
      created_at: 1690209362003
    }
  ]
}

Unredaction rule for Border endpoint:

{
  path: '/search-users',
  method: 'post',
  collections: [
    {
      name: 'user',
      entityIdPath: '$[*].id',
      globalEntityId: true,
      entityErrorCorrectionFieldPath: '$[*].name',
      strategies: [{ path: '$.name', isErrorCorrectionField: true }],
    },
  ],
}

For now unredacationRule.path should match searchEndpoint URL path part so Border can properly use it.

Border response:

{
  users: [
    {
      id: 'user-0.6213436337059335',
      name: 'John',
      car: 'Ford',
      created_at: 1690209362003
    }
  ]
}