InCountry Border Documentation
This guide provides information about InCountry Border and how you can use it for implementation of data residency services without using REST API or SDK.
Overview
InCountry Border is a component of InCountry Ecosystem which allows you to take advantage of data residency services without developing any solutions with InCountry SDK. This component integrates between your customer-facing frontend and system backend. It provides on-the-fly redaction and unredaction of regulated or sensitive data transmitted during execution of API requests.
Data redaction and unredaction rules are specified within a YAML file. This configuration file is further deployed on the InCountry Border side.
Pre-requisites
Before you proceed to using InCountry Border you need to do the following:
- Identify all REST endpoints which you want to integrate with InCountry Border and create a list of them.
- Identify data that is passed or retrieved from each REST endpoint.
- Identify which data is regulated and which data can be used as is. Regulated data will further follow the redaction and unredaction rules which InCountry sets up for you.
Requirements for REST requests
InCountry Border imposes several requirements for execution of REST requests, as follows:
- Data requests are performed through RESTful API. Data is communicated in the JSON format between the customer-facing frontend and system backend of the application.
- For the correct execution of modification (create, update) requests, there must be a field (error correction field) with a random output redaction strategy on it.
id
of a newly created record must be present in the response to the request for creating a new record.id
of the record should be present in the body of a request or in the URL request (for example,POST /users/1
) in data requests for updating records.id
of the record should be present in the URL request for data requests for deleting records.- Data requests should be performed in the JSON format (
content-type: application/json
) if data redaction should be applied. Other data formats will be ignored and will be passed as is.
Data redaction flow
Data redaction is automatically applied by InCountry Border according to the configured data redaction rules.
Let’s imagine a situation. We have a system which allows you to manage media content in the format of pages. The system provides the basic capabilities for managing these pages (like creating, reading, updating or deleting). The page
entity includes both regulated and non-regulated data, as follows:
{
"title": "Page title",
"author_name": "Author name", // PII data
"author_email": "Author email", // PII data
"created_at": "Date of creation"
}
Within this data structure we want to redact the two data fields with regulated data author_name
and author_email
. The rest of fields should be passed as is without redaction.
We publish a newly created page by sending a POST request to the REST endpoint, as follows:
POST /pages
{
"title": "How to follow compliance requirements",
"author_name": "John Doe",
"author_email": "john.doe@gmail.com",
"created_at": "2000-09-23T08:06:07.217Z"
}
To correctly handle data redaction of these two fields, InCountry Border will apply two strategies to the executed data request to the /pages
endpoint. In the current example we have two regulated fields, for each field a specific strategy is applied. Each strategy may take additional parameters. In our case, the path and strategy parameters are applied, as follows:
- The
path
parameter is ajsonpath
string which defines the field in the request body. - The
strategy
is a tokenization algorithm whish is applied to the field according to the redaction policy.
InCountry Border supports the following tokenization algorithms:
alphaNumeric
applies an alpha-numeric string.alphaNumericLowerCase
applies the alpha-numeric string in lower case.alphaPrepended
applies a prefix comprised of a single letter.email
applies an email-valid string, e.g. dsf34fsdf@redactedemail.com.plain
forwards the original value.one
applies '1' ( a single digit).zero
applies '0' (a single digit).numeric
applies a random numeric value of the length equal to the original value.dateISO
applies a random date in the ISO format.defaultDateISO
applies a random date in the default ISO format (1970-01-01T00:00:00Z
).fixed
applies any hardcoded value.
Once InCountry Border captures this POST request, it applies strategies and modifies the request, as follows:
{
"title": "How to follow compliance requirements",
"author_name": "eifsd42fds",
"author_email": "nvstwe31ds@incountry.com",
"created_at": "2000-09-23T08:06:07.217Z"
}
Along with tokenization, InCountry Border also performs encryption of regulated data prior to its saving to InCountry Platform.
The key1
- key10
attributes can be used for record lookup and must be explicitly defined in the Searchable section within the configuration file. These keys may contain non-regulated data too.
{
"record_key": "UUID",
"body": "encrypted(PII object)",
"key1": "PII_field1_token",
...,
"keyN": "non-regulated data field",
}
The key1
and key2
attributes may be empty if the author_name
and author_email
fields are not marked as searchable.
Upon successful data saving, InCountry Border can further forward a tokenized request to the customer’s backend. At this point the regulated data is stored in the encrypted format in InCountry Platform and tokenized data is sent to the customer’s backend. The backend returns an identifier of a newly created record.
InCountry Border intercepts this response and updates the profile_key
with the actual value from the customer’s database.
Following up on this example, let’s try to read information about the saved page including the redacted data about it. In our case the page identifier equals to page1
, so our request will be as follows:
GET /pages/page1
Data that is stored about this page in InCountry Platform is the following:
{
"record_key": "some UUID",
"body": "encrypted({author_name, author_email})",
"key1": "eifsd42fds",
"key2": "nvstwe31ds@incountry.com"
}
Data that is stored about this page in the customer’s database is the following:
{
"pageId": "page1",
"title": "How to follow compliance requirements",
"author_name": "eifsd42fds",
"author_email": "nvstwe31ds@incountry.com",
"created_at": "2000-09-23T08:06:07.217Z"
}
InCountry Border supports the three basic types of backend-frontend interactions:
- mutation (POST, PUT)
- deletion (DELETE)
- reading (GET)
If you need to mutate an entity which includes regulated data, InCountry Border redacts this query and saves regulated data into InCountry Platform and forwards the data request with tokenized data.
If you need to get regulated data, InCountry Border unredacts data stored in the customer’s database by using the previously data from InCountry Platform.
Configuration YAML file
note
The configuration YAML file is produced by InCountry team per your request.
The configuration YAML file is used to manage behavior of InCountry Border for data redaction. The file structure looks, as follows:
rules:
country: SG #Country code, which is used to construct PoPAPI request.
supported_countries: #The list of countries where PII redaction/unredaction is allowed
- SG
features: #Additional features list to switch on
- geoip #Determining ip address where the request came from
target: "" #Tenant's API base URL backend where to forward requests
vaultKeyPath: "/v1/secret/data/incountry" #Secret key path in Vault for encryption/decryption
endpoint: "https://sg-mt-01.uat.incountry.io" #PoPAPI endpoint
environmentId: "<env_id>"
oauth:
clientId: "<client_id>"
clientSecret: "<client_secret>"
authEndpoints:
default: "https://auth-apac.uat.incountry.com/oauth2/token"
redactions: #This section describes rules applied to the request on the way from the frontend to the backend.
- path: "/users" #RegExp to match request Path
method: POST #Request method to match
collectionName: "users" #Tenant's collection name. For example users for the users collections. It is used along with Entity ID in order to uniquely identify the document across all the tenant's documents.
entityIdPath: "$.id" #JSONPath for the field containing Entity ID. Entity ID is an identifier for the document in the tenant's DB.
entityErrorCorrectionFieldPath: "$.email" #JSONPath for the field that is used for error correction. Error correction mechanism is described in Border Architecture document.
searchable: #Make the following InCountry storage fields searchable by defined fields values
key2: "$.name" #JSONPath for the field containing searchable value
key3: "$.email"
strategies: #An array of objects, that describe modifications (redactions) applied to request fields
- strategy: alphaNumeric #A name of tokenization strategy, applied to the matching field
path: "$.name" #JSONPath for the field. According to this path strategy is applied
- strategy: email
path: "$.email"
- path: "/users/.+"
method: PATCH
collectionName: "users"
entityIdPath: "$.id"
entityErrorCorrectionFieldPath: "$.email"
searchable:
key2: "$.name"
key3: "$.email"
strategies:
- strategy: alphaNumeric
path: "$.name"
- strategy: email
path: "$.email"
- path: "/users/(.+)"
method: DELETE
isDeleteRequest: true #Boolean flag that indicates request as a delete request. Unlike the other requests, for delete request we want to forward request to the customer's backend first.
collectionName: "users"
unredactions: #This section describes rules applied to the response from the tenant's backend
- path: "/users/(.+)"
method: GET
collections: #Each response potentially may contain data from multiple collections. For example, when you fetch posts data, it may also include user data (post's author). Each collection item must describe all the needed unredactions for the matching path.
- name: "users"
entityIdPath: "$.id"
entityErrorCorrectionFieldPath: "$.email"
strategies:
- path: "$.name"
- path: "$.email"
isErrorCorrectionField: true #Means that this field will be used for error correction. Error correction mechanism is described in Border Architecture document.
- path: "/users"
method: GET
collections:
- name: "users"
entityIdPath: "$[*].id"
entityErrorCorrectionFieldPath: "$.email"
strategies:
- path: "$.name"
- path: "$.email"
isErrorCorrectionField: true
- path: "/users"
method: POST
collections:
- name: "users"
entityIdPath: "$.id"
entityErrorCorrectionFieldPath: "$.email"
strategies:
- path: "$.name"
- path: "$.email"
isErrorCorrectionField: true
The YAML file is comprised of the three data groups, as follows:
- meta information
- redactions
- unredactions
Meta information
Parameter | Description |
---|---|
country | Country code which is used to construct a PoPAPI request. |
target | Address of the customer’s backend which requests are forwarded to. |
endpoint | URL address of PoPAPI server. |
clientId | Client ID which was issued on the InCountry Portal for a specific environment. |
clientSecret | Client secret which was issues on the InCountry Portal for a specific environment. |
environmentId | Identifier of the environment for accessing the specified PoPAPI server. |
Redactions
This group of parameters defines the rules applied to the data request when passing it from the customer-facing frontend to the application’s backend. Each rule may include the following fields:
Parameter | Description |
---|---|
path | A regular expression to match the request’s path. |
method | A request method that is matched by InCountry Border. |
entityIdPath | JSONPath for the field containing an Entity ID. The Entity ID is an identifier for the document in the customer's database. |
entityErrorCorrectionFieldPath | JSONPath for the field that is used for error correction. |
collectionName | A customer's collection name. For example pages for the pages collection. It is used along with Entity ID in order to uniquely identify a record throughout all the customer's records. |
isDeleteRequest | A boolean flag that marks a request as a delete request. Unlike the other requests, for running a delete request we need to forward a request to the customer's backend first. |
strategies | An array of objects that define modifications (redactions) applied to the request fields. |
strategy | A name of a tokenization strategy that is applied to the matched field. |
path | JSONPath for the field. According to this path a corresponding strategy is applied. |
Unredactions
This group of parameters defines the rules applied to the response from the customer's backend. Each rule may include the following fields:
Parameter | Description |
---|---|
path | A regular expression to match the request’s path. |
method | A request method that is matched by InCountry Border. |
collections | Each response may potentially contain data from multiple collections. For example, when you fetch pages data, it may also include authors data (author of a page). Each collection item must describe all the needed unredactions for the matching path. |
Request structure and modifications
When integrating InCountry Border into your system, please ensure that the application’s frontend points to InCountry Border instead of the application’s backend, so all the requests are passed through InCountry Border.
An example of URL for the tenant’s application frontend looks as follows:
https://{border_url}/x-inc-{tenant}/{tenant_rest_enpoint}
For a single-tenant deployment, you can specify the tenant identifier. In this case, you do not need to specify the header or postfix.
https://{border_url}/{tenant_rest_enpoint}
The URL is comprised of the following parts:
{border_url}
: URL of InCountry Border you want to use.{tenant}
: Name of the tenant (customer).{tenant_rest_endpoint}
: REST endpoint (for example/pages/page1
).
Alternatively, you can define a tenant through the X-INC-CLIENT-ID
header when performing requests through InCountry Border. This approach will not work when using the CORS policy for handling requests, because custom headers are not allowed in the preflight requests. In this case, the URL to point for the tenant’s application frontend will be as follows:
https://{border_url}/{tenant_rest_enpoint}
Let’s imagine a case when the country of data origin is Singapore. The name of the tenant is 'incountry'. We also want to use InCountry Border deployed in Singapore and its endpoint, as follows:
https://sg-proxy-mt-01.uat.incountry.io
InCountry Border is using Singapore PoPAPI
https://sg-mt-01.uat.incountry.io
The resulting URL for endpoint that should be used by the tenant’s application frontend will be as follows:
https://sg-proxy-mt-01.uat.incountry.io/x-inc-incountry/pages/page1
or
https://sg-proxy-mt-01.uat.incountry.io/pages/page1
or
https://sg-proxy-mt-01.uat.incountry.io/pages/page
For the second URL you need to use the request header X-INC-CLIENT-ID=incountry
.
You need to replace all the existing REST endpoints that the application frontend uses with the modified URL’s passing requests through InCountry Border.
What’s next
To integrate InCountry Border into your application, please follow these steps:
- Collect the requests that should be handled by InCountry Border.
- Elaborate the data model and regulated data that should undergo redaction/unredaction.
- Contact the InCountry team to construct a configuration YAML file.
- Wait until the configuration YAML file is deployed on the InCountry Border side.
- Replace the addresses of REST endpoints that will be handled through InCountry Border.
- Verify data requests in your application.