Web Services Proxy

Introduction

InCountry Web Services supports the unredaction of search requests that deal with regulated and sensitive data. Such search requests are performed against two data sources:

1. InCountry platform that stores regulated data

2. application backend that queries non-regulated data from its database

This documentation will explain how you can integrate search capabilities into your web application and what you need to implement on your side, as well as technical limitations that may arise.

Solution Overview

This solution allows users to search data by regulated fields stored in InCountry Vault, as well as by non-regulated fields stored in the application database.

Web Services first searches for appropriate records in InCountry Vault by regulated fields and then sends the list of found records' identifiers combined with the search query by non-regulated fields to the application backend. The application backend should then return a filtered list of entities. In this case, the response from the application backend will contain redacted values (hashes/tokens), they are further unredacted by Web Services using the regular unredaction mechanism.

Application Backend Limitations

The application backend must meet the following requirements:

• It must have a RESTful API that returns and accepts data in the JSON format only.

• It must operate with entities/resources and return a list of entities/resources having some kind of ID as a response to a search query.

• It must have an endpoint for search authentication check with token-based authentication (OAuth, JWT)

Authentication and Authorization in Application Backend

Web Services does not have a mechanism to check the authentication of the incoming request, so the application backend is responsible for the authentication check. Additionally, Web Services does not have an authorization mechanism for verifying user’s access to a specific record, so the application backend is responsible for the authorization check as well.

Workflow

1. The client (browser) sends a search request to Web Services

   The user wants to find all customers with first name “John” (regulated field) and a car “Ford” (non-regulated field). Request from client to Web Services:

   // Request
   // POST https://<WEB_SERVICES_TARGET_ENDPOINT>/users-search
   // Authorization: TOKEN
   {
     "first_name": "John",
     "car": "Ford"
   }

2. Web Services verifies that the client is identified by calling authentication endpoint at application backend:

   // Request
   // POST https://backend.com/api/search-auth
   // Authorization: TOKEN

3. Auth was by the application backend was successful:

   // Response
   // => HTTP 200 OK
   note

   • The application backend should return a non-2xx (>=400) HTTP response code in case of an unauthorized request.

4. (Not applicable to this example but also shown on diagram) Auth by the application backend was unsuccessful and Border returns 400 Bad request.

5. Web Services looks up for all records in InCountry where "first_name" is "John" and gets the list of record identifiers: "id1", "id2", "id3"

6. Web Services adds the list of identifiers to search criteria and sends a request to application backend with parameters:

   // Request
   // POST https://backend.com/api/users/search
   // Authorization: <TOKEN>
   {
     "ids": ["id1", "id2", "id3"],
     "car": "Ford"
   }
   note

   If search in InCountry returns no matching records than empty list of identifiers will be added to the search criteria.

   {
     "ids": [],
     "car": "Ford"
   }

7. The Web Services receives response from the application backend with list of found entities with hashed values:

   // Response
   // => HTTP 200 OK
   {
     "users": [
       {
         "id": "id1",
         "first_name": "4afc0efcf4190920afb8a5cb2ba",
         "last_name": "93133f1737d31c9aea0c6677d0b",
         "car": "Ford"
       },
       {
         "id": "id3",
         "first_name": "4afc0efcf4190920afb8a5cb2ba",
         "last_name": "920afb8a5cb2bade93133f17",
         "car": "Ford"
       }
     ]
   }
   note

   • The application backend should return a non-2xx (>=400) HTTP response code in case of an unauthorized request.
   • The application backend should only return records that the client is authorized to access.

8. The Web Services transforms the response from the application backend, replacing hashed values with clear text and returns the list of found entities:

   // Response
   // => HTTP 200 OK
   {
     "users": [
       {
         "id": "id1",
         "first_name": "John",
         "last_name": "Dow",
         "car": "Ford"
       },
       {
         "id": "id3",
         "first_name": "John",
         "last_name": "Smith",
         "car": "Ford"
       }
     ]
   }

Configuring search

Prerequisites

To create setup similar to described in workflow section you will need:

1. Enabled Web services. See documentation on how to enable Web services.

2. Scheme with indexed string fields. See documentation on how to manage schemas

   Like in workflow example first_name and last_name fields are specified.

   

3. Let's assume a redaction rule to for creating new user were also created before.

   

4. And also unredaction rule to view all users is there.

   

Adding Search rule

Search redaction rule can be using same "ADD REDACTION RULE" button.

Adding Unredaction Rule for Search Results

To unredact search results separate unredaction rule similar one already is needed.