Global Search
The code sample below shows how to get the first page of records from multiple points of presence in parallel. In case you need more than one page, please refer to Read Many Records for the example of how to iterate over the list of pages using the same helper methods.
The following code uses fetch API that is available starting from Node.js version 18. In case you are using the older version or the runtime without fetch support, the implementation can be changed in the request function.
For every RestAPI you want to include in the list of searchable vaults, you have to create a set of credentials and add them in the constant LOCATIONS in the code sample below.
By default, this code requests the first page from every RestAPI and combines results into one array. If you want to send any search criteria to read specific records, please change the value of the FIND_PAYLOAD constant. For the possible values, please refer to the search API documentation.
To run the script using Node.js, assuming you saved the source code in the file named global_search.js, please run the following command in the terminal:
node global_search.js
In case the script successfully received the list of records, it prints no additional information rather than a JSON array. So results could be written into a file using the following command:
node global_search.js > results.json
Also, please note, that in case failure this script logs client credentials and some other information using debug output. If this behavior is unwanted, you can increase the log level to suppress the debug messages using the method suitable for your JS runtime.
The following code sample is simplified example of how to read records from multiple points of presence. For the sake of simplicity some aspects, like advanced error handling, were omitted. In the production code, you will want to handle all the errors from each promise started in the entry point and maybe use Promise.allSettled API to make sure all the vaults returned something.
/**
* @typedef {object} LocationDefinition
* @property {string} url - Actual URL of the RestAPI
* @property {string} authProviderUrl - URL of OAuth2 provider
* @property {string} envId - Target environment ID
* @property {string} country - Two letter country code
* @property {string} clientId - Created on InCountry portal
* @property {string} clientSecret - Created on InCountry portal
* @property {string} [audience] - Space separated list of URLs of appropriate resources in a RestAPI.
* Like API and Storage. In case missing, it will be generated using a common template
*/
/**
* @typedef {object} FindPayload
* @property {Object.<string, any>} filter - A set of search criteria
* @property {Object.<string, any>} options - A list of options for the find endpoint
/**
* A list of locations to search
* To a create new service and get the credentials please refer to the documentation
* https://docs.incountry.com/portal/managing-services/#creating-a-new-rest-api-service
*
* @constant
* @type {Array.<LocationDefinition>}
*/
const LOCATIONS = [
{
url: "<hostname of an appropriate PoP>/api/records/find",
authProviderUrl: "<hostname of an OAuth2 provider>/oauth2/token",
envId: "<env id>",
country: "<two letter country code>",
clientId: "<client identifier>",
clientSecret: "<password>",
},
];
/**
* Find payload
* Consists of filters and options. For the examples and use cases please refer to the documentation
* https://docs.incountry.com/data-residency-as-a-service/search-rest-api/
*
* @constant
* @type {FindPayload}
*/
const FIND_PAYLOAD = {
filter: {},
options: {},
};
(async function (locations, payload) {
const results = (
await Promise.all(locations.map((location) => search(location, payload)))
).flat();
console.log(JSON.stringify(results));
})(LOCATIONS, FIND_PAYLOAD);
// Below this line, there are internal functions to execute find requests. In order to alter the results set
// please modify FIND_PAYLOAD object above
// --------------------------------------------------------------------------------------------------------------------
/**
* Search a given RestAPI
*
* @param {LocationDefinition} location
* @param {FindPayload} payload - An object that will be passed as a payload to the find endpoint
*/
async function search(
{ url, authProviderUrl, country, envId, clientId, clientSecret, audience },
payload,
) {
const accessToken = await authenticate(
authProviderUrl,
envId,
clientId,
clientSecret,
audience || generateAudience(country),
);
return readRecords(url, envId, accessToken, { ...payload, country });
}
/**
* Get access token for a given location
*
* For more information please refer to the documentation
* https://docs.incountry.com/data-residency-as-a-service/authentication/
*
* @param {string} url - OAuth2 server URL
* @param {string} envId - Target environment ID
* @param {string} clientId - Client identifier created on InCountry portal
* @param {string} clientSecret - Password created on InCountry portal
* @param {string} audience - A list of URLs of appropriate resources in a RestAPI. Like API and Storage
* @returns {Promise.<string>} Opaque access token
*/
async function authenticate(url, envId, clientId, clientSecret, audience) {
const { access_token: accessToken } = await request(
url,
{
grant_type: "client_credentials",
response_type: "token",
scope: envId,
audience: audience,
},
{
"Content-Type": "application/x-www-form-urlencoded;charset=UTF-8",
Authorization:
"Basic " +
Buffer.from(clientId + ":" + clientSecret).toString("base64"),
},
);
if (!accessToken) {
console.debug(
`Unable to get an access token from ${url} using`,
clientId,
clientSecret,
audience,
);
throw new Error(
`Unable to get an access token from ${url}. Please check your url and credentials.`,
);
}
return accessToken;
}
/**
* Read an actual list of records from the RestAPI
* For more information and examples please refer to the documentation
* https://docs.incountry.com/data-residency-as-a-service/search-rest-api/
*
* @param {string} url - Actual URL of the RestAPI
* @param {string} envId - Environment ID
* @param {string} accessToken - Opaque access token
* @param {FindPayload} payload - Search criteria and options
* @returns {Promise.<Object.<string, any>>}
*/
async function readRecords(url, envId, accessToken, payload) {
const response = await request(url, payload, {
Authorization: `Bearer ${accessToken}`,
"X-Env-Id": envId,
});
if (!("data" in response)) {
console.debug(
"Unable to execute find request",
url,
envId,
accessToken,
payload,
response,
);
throw new Error("Unable to execute find request to", url);
}
return response.data;
}
/**
* Send HTTP POST request using fetch API
* For this example we will use only POST requests
*
* @param {string} url
* @param {Object.<string, any>} payload - An object that will be serialized and sent to the target endpoint
* @param {Object.<string, string|number|boolean} - A list of HTTP headers
* @returns {Promise.<Object.<string, any>>}
*/
async function request(url, payload, headers) {
payload = headers["Content-Type"]?.startsWith(
"application/x-www-form-urlencoded",
)
? new URLSearchParams(payload)
: JSON.stringify(payload);
const response = await fetch(url, {
mode: "cors",
method: "POST",
cache: "no-cache",
headers: new Headers({
"Content-Type": "application/json",
...headers,
}),
body: payload,
});
if (response.status !== 200) {
console.debug(
"Fail to execute an HTTP request",
url,
payload,
headers,
response,
);
throw new Error(
`Unable to execute request to ${url}. Please check the payload and the headers.`,
);
}
return response.json();
}
/**
* Generate audience using our generic template
*
* @param {string} country - Two letter country code
* @returns {string}
*/
function generateAudience(country) {
return `https://${country}-restapi-mt-01.api.incountry.io https://${country}-mt-01.api.incountry.io`;
}