Web Services Endpoints

You can manage endpoints that are handled by Web Services for redacting and unredacting regulated data. An endpoint represents a collection of rules describing how a request or response looks like and which fields must be redacted or unredacted and which fields must be passed as-is.

Web Services supports two data flows when:

1. It redacts ingress data: converts input values (clear-text values) into tokenized values and replaces them within request payloads.
2. It unredacts egress data: swaps tokenized values with their original clear-text values within response bodies.

Web Services handles regulated data based on redaction and unredaction rules that describe the structure of request payloads and response bodies and regulate fields that contain sensitive values. Web Services intercepts web requests with their payloads or responses and acts accordingly to the configured redaction or unredaction rules. The intercepted regulated values are either redacted and saved to the InCountry platform (in redaction rules) or queried from the InCountry platform and swapped in responses (in unredaction rules). For example, for the POST, PUT, and UPDATE methods, you can apply a redaction rule, for the GET method you can apply an unredaction rule.

Depending on the value format, you need to apply an appropriate redaction algorithm to regulated values processed by Web Services. This greatly depends on your system architecture and handled values, as well as validation rules.

To preserve the search capabilities, you need to explicitly indicate what values in the payload should be treated as searchable. In this case, regulated values will be additionally saved to searchable fields where the InCountry platform can easily look up records against several searchable fields.

Redaction and unredaction rules deal with collections, so-called objects whose data undergoes redaction or unredaction. Web Services uses collections to produce unique hash values and identify the applicable redaction or unredaction strategies in the data communication pipeline.

Creating Web Services endpoints

1. Open the environment where you want to create a Web Services endpoint.
2. In the navigation sidebar, select the country which a Web Service is attributed to.
3. On the list of services, locate the service of the Web Services type.
4. Click the Manage endpoints icon. Portal opens the list of Web Services endpoints.
5. Click Add Endpoint.
6. It the Properties section, specify the following information:
   • Name - enter the name of the Web Services endpoint.
   • Existing REST URL - specify the URL of the application that handles regulated data.
   • Entity IDs are unique - check the box only when the entity id within the payload is a completely unique value that cannot be duplicated.
   • Data masking - check the box to enable the data masking for unredaction rules. By default, Web Services automatically unredacts data in responses, with this option activated Web Services will return masked values for users requests data outside the country of origin. So original clear-text values will not be reveled if requested outside the country of origin. When configuring unredaction rules, you will be able to define the value masking options.
   • CORS settings - check the box to override CORS settings. Expand the CORS Settings section and specify the required Access-Control header. This header will be applied to all requests handled through Web Services.
7. Click Add redaction rule. Specify redaction rules that will regulate how Web Services redacts regulated data passing through the endpoint. For the details on how to configure a redaction rule, please see the Managing redaction rules section. You can add multiple rules to redact data passing through different endpoints.
8. Click Add unredaction rule. Specify unredaction rules that will regulate how Web Services unredacts regulated data passing through the endpoint. For the details on how to configure an unredaction rule, please see the Managing unredaction rules section. You can add multiple rules to redact data passing through different endpoints.
9. When complete, click Add Endpoint.
10. On the Endpoint review and confirmation page, review the Web Services endpoints you have specified.
11. Enter the verification code and click Confirm.

info

Once you have saved the Web Services endpoint, InCountry Portal will output information about the Web Services endpoint through which you need to pass requests within your web application.
You need to locate the Target endpoint value, which will look like
https://se-proxy-mt-01.incountry.io/x-inc-55898eff-361a-XXXX-XXXX-4b755d0afe8b
In the source code of your web application, you need to replace the web application URL with this target endpoint for all endpoints that you configured in the Web Services endpoint.
For example, you have the POST https://socialposts.com/api/authors/ endpoint, you need to make it look like https://se-proxy-mt-01.incountry.io/x-inc-55898eff-361a-XXXX-XXXX-4b755d0afe8b/api/authors/.

Warning

Please ensure that you use the correct country for handling regulated data. The ISO country code is specified in the target endpoint address. If the incorrect country is specified, Web Services will not be able to unredact regulated data when querying from the InCountry Vault where these records do not exist. The redaction of regulated data may result in the situation when you save regulated country in the country different from the country of origin.

Managing redaction rules

InCountry Data Residency-as-a-Service provides four different types of redaction rules:

• Default rule
• Extended rule
• Salesforce Cross-Border
• Search request

By clicking on the Add redaction rule button, the following window will open. Depending on your selection, a different redaction rule editor will be displayed.

Default Rule

1. To add a default redaction rule, сlick Add redaction rule, then select Default rule.

2. In the Add redaction rule form, specify the following information:

   1. HTTP request method - select the request type, as follows:
      1. GET - used to retrieve data.
      2. POST - used to submit/create new data.
      3. PUT - used to submit/create new data.
      4. PATCH - used to update data.
      5. DELETE - used to delete data.

3. Path relative to Existing REST URL - enter the URI path to the endpoint and append the regular expression if needed to handle cases when the slash is either present or not in the request.

4. Request payload example (JSON) - enter the request payload that you want to redact when passing through Web Services to the application backend.

5. Response body example (JSON) - enter the response body that the application backend returns to the response.

6. After configuring these settings, click Add rule.

7. Click the Advanced settings icon. Here you can select the appropriate operating mode for this endpoint, as follows:

   1. Processing & Storage - ingress regulated data is saved to the InCountry platform and then it is redacted and passed to the endpoint.
   2. Processing - ingress regulated data is redacted and passed to the endpoint without its saving on the InCountry platform.

   

8. On Step 1: Schema select Schema for this redaction rule or create a new one. InCountry allows you to define schema of your records for more convenient execution of requests against regulated values.

9. At the Step 2: Fields to redact step, you need to specify all the fields from the request payload that contain regulated values that you want to redact. For each field you want to redact, specify the following information:

   • Path - specify the JSON path to the field carrying a regulated value that should be redacted. InCountry Portal also automatically parses the request payload, so you can select the necessary field from the dropdown box.

   • Algorithm - select the appropriate algorithm to redact the regulated value from the available options. Please use the algorithm that matches the format of the original value.

     • alphaNumeric - applies an alphanumeric hash to a string containing letters and numbers. The produced alphanumeric string varies during every redaction.
     • alphaNumericLowerCase - applies a lower-case alphanumeric hash to a string containing letters and numbers.
     • alphaPrepended - applies a prefix comprising a single letter.
     • email - applies an email-pattern string, e.g. dsf34fsdf@redactedemail.com. The produced email-pattern string varies during every redaction.
     • plain - forwards the original value.
     • one - applies '1' ( a single digit).
     • zero - applies '0' (a single digit).
     • numeric - applies a random numeric value of the length equal to the original value.
     • dateISO - applies a random date in the ISO format.
     • defaultDateISO - applies a random date in the default ISO format (1970-01-01T00:00:00Z).
     • fixed - applies any hardcoded value. If you select this option, in the Value box, enter the value that should be applied by default.
     • emailPersistent - applies an email-pattern string that remains the same for the same email address during every redaction.
     • alphaNumericPersistent - applies an alphanumeric hash that remains the same for the same alphanumeric string during every redaction.
     • masking - applies value masking (email or alphanumeric), that is, this strategy may leave some characters in the value, but replaces most with selected characters beyond recognition, e.g. Alan Smith will be masked by default as Ala****** Smi****** and alansmith@example.com will be masked by default as ala******@example.com. The number of remaining and masking characters can be further configured.

   • Length - enter the length for the redacted value if your system performs some validation of value length, otherwise leave the default length.

   • Indexed field - select the record’s field that you created in Schema which will be used to store a searchable value of the record on the InCountry platform.

   

10. At the Step 3: Entity identification step, specify the following information:

    • Prefix to add to non-unique/unqiue ID - enter a meaningful and unique name to the collection that will allow endpoint to correctly differentiate between entities even when they have the same identifier (this step is necessary, if the Entity IDs are unique checkbox is not checked, i.e. the entity id within the payload is a non-unique value that can be duplicated).

    • Path to non-unique/unique ID - ensure that the regular expression matches the path of your request.

    • Randomly tokenized field - enter the path to the record identifier that is returned within the response to this request.

      

11. Finally, at the Step 4: Data Loss Prevention (DLP) excluded paths step, you need to specify particular paths from the request body that will be exempted from automatic data redaction. Let us remind you that InCountry Web Services could automatically identify and redact potential personally identifiable information (PII) data within the payload as it crosses borders even if it is not configured in the fields to redact.

    

Extended Rule

1. To add an extended redaction rule, сlick Add redaction rule, then select Extended Rule. With the Extended Rule option, you can finely adjust the configuration for the rule and also set up batch creation or updating of records, which isn't possible with the Default Rule.

2. In the Add redaction rule form, specify the HTTP request method, path relative to existing REST URL, request payload and response body JSON examples, everything is exactly the same as for the default rule. After configuring these settings, click Add rule. In the Extended Rule, a request can contain several different entity types. For each such entity, they should be created within the redaction rule. The first entity is automatically generated.

3. In the entity configuration, click the Advanced settings icon. Here you can select the appropriate operating mode for this endpoint e.g. Processing & Storage or only Processing.

4. On Step 1: Entity identification, fields for identification are separated for the request and response. In the request section, you must specify the path to a single element as Entity path, followed by the path to the element ID inside it as Path to non-unique/unique ID. Similarly, you need to specify the path to the element in the response and the path to the ID inside the element. This is designed to facilitate the configuration of batch creation or updating of records. If you don't have a batch request, but a regular flat request, you can specify an empty JSON path $. in the field for determining the path to the element.

   

   For example, if the request body is the following JSON:

   {
      "RequestSkills": [{
         "RequestId": "requestId1",
         "proficiency": "writing",
         "ability": "high"
      }, {
         "RequestId": "requestId2",
         "proficiency": "speaking",
         "ability": "middle"
      }]
   }

   and the response body is:

   {
      "ResponseSkills": [{
         "ResponseId": "responseId1",
         "proficiency": "writing",
         "ability": "high"
      }, {
         "ResponseId": "responseId2",
         "proficiency": "speaking",
         "ability": "middle"
      }]
   }

   then request entity path is $.RequestSkills[*] and path to ID is $.RequestId, and response entity path is $.ResponseSkills[*] and path to ID is $.ResponseId.

5. On Step 2: Schema select Schema for this redaction rule entity or create a new schema. The InCountry platform allows you to define schema of your records for more convenient execution of requests against regulated values.

   

6. At the Step 3: Fields to redact step, specify the following information:

   • Path - specify the JSON path to the field carrying a regulated value that should be redacted. InCountry Portal also automatically parses the request payload, so you can select the necessary field from the dropdown box.

   • Algorithm - select the appropriate algorithm to redact the regulated value from the available options. Please use the algorithm that matches the format of the original value.

     • alphaNumeric - applies an alphanumeric hash to a string containing letters and numbers. The produced alphanumeric string varies during every redaction.
     • alphaNumericLowerCase - applies a lower-case alphanumeric hash to a string containing letters and numbers.
     • alphaPrepended - applies a prefix comprising a single letter.
     • email - applies an email-pattern string, e.g. dsf34fsdf@redactedemail.com. The produced email-pattern string varies during every redaction.
     • plain - forwards the original value.
     • one - applies '1' ( a single digit).
     • zero - applies '0' (a single digit).
     • numeric - applies a random numeric value of the length equal to the original value.
     • dateISO - applies a random date in the ISO format.
     • defaultDateISO - applies a random date in the default ISO format (1970-01-01T00:00:00Z).
     • fixed - applies any hardcoded value. If you select this option, in the Value box, enter the value that should be applied by default.
     • emailPersistent - applies an email-pattern string that remains the same for the same email address during every redaction.
     • alphaNumericPersistent - applies an alphanumeric hash that remains the same for the same alphanumeric string during every redaction.
     • masking - applies value masking (email or alphanumeric), that is, this strategy may leave some characters in the value, but replaces most with selected characters beyond recognition, e.g. Alan Smith will be masked by default as Ala****** Smi****** and alansmith@example.com will be masked by default as ala******@example.com. The number of remaining and masking characters can be further configured.

   • Length - enter the length for the redacted value if your system performs some validation of value length, otherwise leave the default length.

   • Indexed field - select the record’s field that you created in Schema which will be used to store a searchable value of the record on the InCountry platform.

   

7. At the Step 4: Randomly-tokenized field step, specify the randomly-tokenized field for the entity both for the request and response as Randomly-tokenized field.

   

   If necessary, create another entity and configure it in a similar way.

Salesforce Cross-Border Rule

This redaction rule is specifically designed for the Salesforce package to help migrate the data from the local Salesforce org to the global Salesforce org. Under the hood of this rule, we implemented support for the Salesforce upsert records using sObject collections method.

1. To add a Salesforce Cross-Border redaction rule, сlick Add redaction rule, then select Salesforce Cross-Border.

2. In the Add redaction rule form, specify the following information:

   1. HTTP request method - select the request type, as follows:
      1. GET - used to retrieve data.
      2. POST - used to submit/create new data.
      3. PUT - used to submit/create new data.
      4. PATCH - used to update data.
      5. DELETE - used to delete data.

3. Path relative to Existing REST URL - enter the URI path to the endpoint and append the regular expression if needed to handle cases when the slash is either present or not in the request. Also, here you should provide the following type of URLs: /services/data/vXX.X/composite/sobjects/SobjectName/ExternalIdFieldName.

4. Request payload example (JSON) - enter the request payload that you want to redact when passing through Web Services to the Salesforce application backend.

5. Response body example (JSON) - enter the response body that the Salesforce application backend returns to the response.

6. After configuring these settings, click Add rule.

7. Click the Advanced settings icon. Here you can select the appropriate operating mode for this endpoint, as follows:

   1. Processing & Storage - ingress regulated data is saved to the InCountry platform and then it is redacted and passed to the endpoint.
   2. Processing - ingress regulated data is redacted and passed to the endpoint without its saving on the InCountry platform.

   

8. On Step 1: Salesforce Cross-Border parameters select parameters for this redaction rule.

   1. Records path - the path to the list of sObjects.
   2. allOrNone parameter - the path to the allOrNone flag that indicates whether to roll back the entire request when the creation of any object fails (true) or to continue with the independent creation of other objects in the request. The default is false.
   3. External ID path - the JSON path to the external IDs of sObjects.
   4. Update by External ID - the boolean flag, and if checked, then in the next request with external ID, the records will be updated and not rewritten.
   5. Search path - is an optional parameter which refers to the URL containing the :externalId placeholder, which points to an endpoint utilizing a specialized error-correction method specific to certain Salesforce services. Request to this URL should return a JSON body. Within this body, the external ID value should be found under the specified JSON path in the "Randomly-tokenized field" section. For example, https://org2orgdemotarget3.my.salesforce.com/services/data/v58.0/composite/sobjects/Contact/:externalId

9. At the Step 2: Fields to redact step, you need to specify all the fields from the request payload that contain regulated values that you want to redact. For each field you want to redact, specify the following information:

   • Path - specify the JSON path to the field carrying a regulated value that should be redacted. InCountry Portal also automatically parses the request payload, so you can select the necessary field from the dropdown box.
   • Algorithm - select the appropriate algorithm to redact the regulated value from the available options. Please use the algorithm that matches the format of the original value. For more information about algorithms, see in the Default Rule section above.
   • Length - enter the length for the redacted value if your system performs some validation of value length, otherwise leave the default length.
   • Indexed field - select the record’s field which will be used to store a searchable value of the record on the InCountry platform.

10. At the Step 3: Entity identification step, specify the following information:

• Prefix to add to non-unique ID - enter a meaningful and unique name to the collection that will allow endpoint to correctly differentiate between entities even when they have the same identifier (this step is necessary, if the Entity IDs are unique checkbox is not checked, i.e. the entity id within the payload is a non-unique value that can be duplicated).

• Path to non-unique ID - ensure that the regular expression matches the path of your request.

• Randomly tokenized field - enter the path to the record identifier that is returned within the response to this request.

  

Search request

The Search request redaction rule allows searching for data by regulated fields stored on the InCountry platform, as well as by non-regulated fields stored in the customer's backend database.

1. To add a Search request redaction rule, сlick Add redaction rule, then select Search request.

2. In the Add redaction rule form, specify the following information:

   1. HTTP request method - select the request type, as follows:
      1. GET - used to retrieve data.
      2. POST - used to submit/create new data.
      3. PUT - used to submit/create new data.
      4. PATCH - used to update data.
      5. DELETE - used to delete data.

3. Path relative to Existing REST URL - enter the URI path to the endpoint and append the regular expression if needed to handle cases when the slash is either present or not in the request.

4. Request payload example (JSON) - enter an example of the search request payload.

5. In the newly added Search request redaction rule, include the following details:

   1. Authentication endpoint - provide the full HTTPS URL of the customer's backend where authentication requests will be sent.
   2. Offset parameter name - specify the request parameter responsible for pagination offset.
   3. Pagination limit parameter name - specify the request parameter responsible for pagination limit.
   4. Default pagination limit - enter the default pagination limit if offset and limit parameters are not specified. In the Mapping for search request criteria section, select the Schema table and its fields that will be utilized for the search.

Managing unredaction rules

While defining the configuration of Web Services, you can set up unredaction rules that regulate how Web Services unredacts regulated data passing through the endpoint. You can add multiple rules to unredact data passing through different endpoints.

1. Click Add unredaction rule.
2. In the Add unredaction rule form, specify the following information:
   1. HTTP request method - select the request type, as follows:
      1. GET - used to retrieve data.
      2. POST - used to submit/create new data.
      3. PUT - used to submit/create new data.
      4. PATCH - used to update data.
      5. DELETE - used to delete data.
   2. Path relative to Existing REST URL - enter the URI path to the endpoint and append the regular expression if needed to handle cases when the slash is either present or not in the request.
3. Click Add rule.
4. At the Step 1: Entity identification step, specify the following information:
   • Entity Prefix - select the prefix (entity name) which the current unredaction rule is attributed to. Web Services will use the prefix to identify records pertaining to a specific entity while unredacting their hashed values. When the response combines multiple entities, you need to properly reference them in the configuration. In this case, you may need to register another entity by clicking Add another entity.
   • Path to non-unique ID - enter the JSON path to the field carrying a non-unique record identifier.
   • Randomly tokenized field - enter the JSON path to field that you redact with an undeterministic tokenization algorithm (email, alphaNumeric, or alphaNumericLowerCase).
5. At the Step 2: Fields unredaction step, click Add unredacted field.
6. Specify the following information for each field you want to unredact:
   • Path - enter the JSON path to the field carrying a regulated value that must be unredacted.
   • Original Path - select the JSON path to the original field carrying a regulated value from the redaction rule.
7. When complete, save the Web Services endpoint.

When your Web Services endpoint deals with unique entity ids (the Entity IDs are unique option is enabled), then you need to specify the path to unique ID and select the randomly-tokenized field.

When you enable the data masking, while configuring unredaction rules, you will be able to define how to mask regulated values when returning them to the application backend residing in the country different from the country of origin. To users coming from the country of origin, Web Services will return the original clear-text value as-is.

Configuring value masking

By default, Web Services automatically unredacts the data in the response body. You can forbid this for users requesting regulated data outside its country of origin. Regulated values will be masked except when requested from the country of origin. Once enabled, you can choose the appropriate masking algorithm within the settings of the unredaction rule.

1. Click Unredact field.
2. For each field you want to mask, specify the following information:
   1. Algorithm - select the masking algorithm that you want to apply. You can select among the following ones:
      1. default - Web Services will use the initial redaction algorithm used for this field in the configured redaction algorithm.
      2. fixed - Web Services will apply the fixed value you enter in the Value field, for example, John Doe can be replaced with REDACTED.
      3. masking - Web Services will apply the masking value to the original clear text value, for example, John Doe can be replaced with J**** D****. For this algorithm, you can specify additional settings:
         1. Type - select the value type (alphanumeric or email).
         2. Masking after N characters - specify the number of characters that will be preserved as is at the string beginning. All the characters then will be masked with the masking character.
         3. Masking character - enter the character that will mask values from the original clear-text value, for example, * (asterisk), @ (ampersand) or X.
         4. Masking length - specify the number of masking characters applied to the string.
         5. Value delimiter - delimiter to separate values with.
   2. Path and Original path - specify the JSON paths as for a generic unredaction rule. See the instructions above.
3. When complete, save the Web Services endpoint.