Guides
StatusPortal

Batch Query API Implementation

Suggest Edits

Summary

Particle Health has the capability to perform Batch Queries. With batch, you can upload a set of patient demographics via a .CSV file or JSON object, and Particle will gradually run those queries during low-traffic periods throughout the day.

Batch may fit your use case if:

  1. Your organization is running scripts programmatically in bulk, and is looking to get information on a n patients at once, without having to manually run n API calls.
  2. Your organization is not limited by speed of data retrieval. We configure the batch queries to run during off-peak hours. Therefore, you can typically expect a batch to complete within 24 hours. If your organization requires data within a faster turn-around-time, our standard query flow is still available to your organization ad-hoc. Read about Particle's Recommended Query Flow, here.

Supported Configurations

Query Types

CCDA, FHIR, and FLAT queries are supported in the batch query flow. Any other query types will return a 403 error.

The project making batch requests must be enabled to run singular queries of that particular type. So if your project does not have FLAT queries enabled, that project may not run FLAT batches. Read about Organization's term definitions, here.

Query Limits

Production is limited to 10,000 queries in a batch. Sandbox is limited to 10 queries in a batch. The maximum request size is 40MB both both environments.

A Project may only have 3 active batches running at any particular time. The List Batches endpoint will return a list of your Project’s batches, in case you wish to verify how many are active at any given point. Should you attempt to create a new batch request while 3 batches are currently active, you will receive a 429 error.


API Flow

  1. Upload a set of demographics to the Create Batch endpoint. Upload either a .CSV document or a JSON object in the POST body, along with a batch parameter that denotes the batch type.
  2. Receive a response with the scheduled queries and query IDs. If any rows are malformed in a submitted .CSV, those errors will be returned in the response. The entire batch must be valid for the process to run!
  3. Each query will run at its scheduledTime . You may opt into a notification to be alerted when each individual query is completed.
  4. As each query is completed, you may fetch the data for that query. You do not need to wait for the entire batch to finish to inspect any individual query in the batch.
    1. Depending on the batch_type specified, the methods to fetch CCDA, FHIR, or FLAT will change accordingly.
  5. When all of the queries in a batch are completed, the batch is marked as COMPLETE and one last notification is sent.

API Methods

Create Batch

POST https://api.particlehealth.com/api/v1/projects/{project_id}/batches is the endpoint to kick off a new batch of queries.

Body Parameters

demographics_json A JSON array containing a set of demographics. Example:

[
  {
    "given_name": "Kam",
    "family_name": "Quark",
    "gender": "FEMALE",
    "date_of_birth": "1979-01-04",
    "postal_code": "11111",
    "purpose_of_use": "TREATMENT",
    "patient_id": "e9e88a62-88d6-4bc7-8677-66b8314a6532"
  },
  {
    "given_name": "John",
    "family_name": "Que",
    "gender": "FEMALE",
    "date_of_birth": "1979-01-04",
    "postal_code": "71116",
    "purpose_of_use": "TREATMENT",
    "patient_id": "e9e88a62-88d6-4bc7-8677-66b8314a6533"
  }
]
  • All parameters in each demographic object must be a string , otherwise the batch queries will not process properly.
  • When submitted, the JSON object must be string-escaped. See Sample JSON Request below.

demographics_csv A .CSV file containing a set of demographics. Example:

given_name,family_name,date_of_birth,gender,postal_code,purpose_of_use,patient_id
John,Doe,MALE,1990-01-01,12345,TREATMENT,20fde9f7-212d-45d2-aa23-b02a42ee6b70
Jane,Doe,FEMALE,1990-01-01,12345,TREATMENT,ec2fbcab-b125-47e5-af22-718b177407cd
  • The first row of the .CSV contains all of the fields to be submitted. See required fields.
  • Please use commas as the delimiter between fields. Do not add spaces before/after commas in the .CSV.
  • If errors are found, the batch will continue and return the .CSV lines and demographics where errors were found. Any properly formatted lines of the CSV will result in a scheduled query.

batch A JSON object containing batch metadata. Example:

{"batch_type":"FHIR_R4"}
  • batch_type can be FHIR_R4, CCDA, or FLAT
  • Must also be string-escaped.

Required Batch Fields

If both demographics_json and demographics_csv are present, the .CSV will be ignored. Regardless of the format, each demographic in the set must include the following:

  • Given Name - given_name
    • Ex. Kam
  • Family Name - family_name
    • Ex. Quark
  • Gender - gender
    • Must be MALE , FEMALE
  • Date of Birth - date_of_birth
    • Ex. 1954-12-01
    • Must follow YYYY-MM-DD format.
  • Postal Code - postal_code
    • Ex. 11111
  • Purpose of Use - purpose_of_use
    • Ex. TREATMENT
  • Patient ID - patient_id
    • Ex. 6a24bbf5-851d-44fc-8470-bea12a6e69cb
    • This must be unique to your project. We recommend a UUID format, but this can be any format of your choosing.

Optional Batch Fields

The following fields are optional, but if included will improve query results.

  • SSN - ssn
    • Ex. 111-00-1234
  • Address Lines - address_lines
    • Ex. 999 Dev Drive
  • Address City - address_city
    • Ex. Brooklyn
  • Address State - address_state
    • Ex. NY
    • Please use the 2-letter state code.

Sample .CSV Request

If you wish to use demographics_csv attach a csv document through the formData http request field. In the example below the file is named batch.csv 

curl --location 'https://api.particlehealth.com/api/v1/projects/{{PROJECT_ID}}/batches' \
--header 'Authorization: Bearer {{TOKEN}}' \
--form 'demographics_csv=@"batch.csv"' \
--form 'batch="{\"batch_type\":\"CCDA\"}"' \

With a batch.csv containing:

given_name,family_name,date_of_birth,gender,address_lines,address_city,postal_code,address_state,purpose_of_use,patient_id,ssn
Kam,Quark,1954-12-01,MALE,999 Dev Drive,Brooklyn,11111,NY,TREATMENT,123451,123-45-6789
Kam,Quark,1954-12-01,MALE,999 Dev Drive,Brooklyn,11111,NY,TREATMENT,123452,123-45-6789
Kam,Quark,1954-12-01,MALE,999 Dev Drive,Brooklyn,11111,NY,TREATMENT,123453,123-45-6789
Kam,Quark,1954-12-01,MALE,999 Dev Drive,Brooklyn,11111,NY,TREATMENT,123454,123-45-6789
Kam,Quark,12-01-1954,MALE,999 Dev Drive,Brooklyn,11111,NY,TREATMENT,123455,123-45-6789

The last line of the provided .CSV is malformed (incorrect date_of_birth). In the response, we see the following error returned. Each value in the .CSV is validated before query processing.

[
    {
        "error": "invalid date_of_birth provided, must be of format YYYY-MM-DD; invalid date_of_birth provided, must be after 1900-01-01",
        "line": 6
    }
]

By editing that line and running the same request with the date in the correct syntax, now we see a sample 200 response:

{
    "batch_id": "37e71d02-4c87-49cb-a6e3-883af09ddc1b",
    "expected_finish_time": "2024-06-24T21:40:20.000000Z",
    "state": "PENDING",
    "query_count": 5,
    "queries": [
        {
            "query_id": "5144ca48-a282-45b7-b816-90aca603bb1a",
            "state": "SCHEDULED"
        },
        {
            "query_id": "a6796705-5b03-42dc-95f8-28dff96c4fb4",
            "state": "SCHEDULED"
        },
        {
            "query_id": "62c70b3f-a05f-47f3-b9e4-9224d2709869",
            "state": "SCHEDULED"
        },
        {
            "query_id": "ba435f90-486f-4b5c-b59f-8c53f71fc6b1",
            "state": "SCHEDULED"
        },
        {
            "query_id": "cb3150f7-11fb-45a6-812c-dfbd5e01e360",
            "state": "SCHEDULED"
        }
    ],
    "batch_type": "CCDA"
}

Sample JSON Request

Similarly, if you wish to use demographics_json , your curl request would look something like this:

curl --location 'https://api.particlehealth.com/api/v1/projects/{{PROJECT_ID}}/batches' \
--header 'Authorization: ••••••' \
--form 'demographics_json="[{\"given_name\":\"Kam\",\"family_name\":\"Quark\",\"gender\":\"FEMALE\",\"date_of_birth\":\"1979-01-04\",\"postal_code\":\"11111\",\"purpose_of_use\":\"TREATMENT\",\"patient_id\":\"e9e88a62-88d6-4bc7-8677-66b8314a6531\"},{\"given_name\":\"John\",\"family_name\":\"Que\",\"gender\":\"FEMALE\",\"date_of_birth\":\"1979-01-04\",\"postal_code\":\"71116\",\"purpose_of_use\":\"TREATMENT\",\"patient_id\":\"e9e88a62-88d6-4bc7-8677-66b8314a6531\"}]"' \
--form 'batch="{\"batch_type\":\"FHIR_R4\"}"'

If this were the sample JSON object we are adding to the body formData 

[
  {
    "given_name": "Kam",
    "family_name": "Quark",
    "gender": "FEMALE",
    "date_of_birth": "1979-01-04",
    "postal_code": "11111",
    "purpose_of_use": "TREATMENT",
    "patient_id": "e9e88a62-88d6-4bc7-8677-66b8314a6532"
  },
  {
    "given_name": "John",
    "family_name": "Que",
    "gender": "FEMALE",
    "date_of_birth": "1979-01-04",
    "postal_code": "71116",
    "purpose_of_use": "TREATMENT",
    "patient_id": "e9e88a62-88d6-4bc7-8677-66b8314a6533"
  }
]

and the response would look like this:

{
    "batch_id": "255113d4-975a-4003-a99a-e4ea72a2f988",
    "expected_finish_time": "2024-06-22T00:00:00.000000Z",
    "state": "PENDING",
    "query_count": 2,
    "queries": [
        {
            "query_id": "a9131a5a-059d-4628-9a7d-cef6f9aae14e",
            "state": "SCHEDULED"
        },
        {
            "query_id": "561937ff-542e-4ece-bbbb-09ddae318974",
            "state": "SCHEDULED"
        },
    ],
    "batch_type": "FHIR_R4"
}

List Batch

GET https://api.particlehealth.com/api/v1/projects/{project_id}/batches

A chronological list of all queries run by your project. Gives a quick snapshot into each batch previously run. Regardless of the batch type you initially created, the response will be the same format.

Sample Request

curl --location 'https://api.particlehealth.com/api/v1/projects/{{PROJECT_ID}}/batches' \
--header 'Authorization: ••••••' \
--form 'demographics_json="[{\"given_name\":\"Kam\",\"family_name\":\"Quark\",\"gender\":\"FEMALE\",\"date_of_birth\":\"1979-01-04\",\"postal_code\":\"11111\",\"purpose_of_use\":\"TREATMENT\",\"patient_id\":\"e9e88a62-88d6-4bc7-8677-66b8314a6531\"},{\"given_name\":\"John\",\"family_name\":\"Que\",\"gender\":\"FEMALE\",\"date_of_birth\":\"1979-01-04\",\"postal_code\":\"71116\",\"purpose_of_use\":\"TREATMENT\",\"patient_id\":\"e9e88a62-88d6-4bc7-8677-66b8314a6531\"}]"' \
--form 'batch="{\"batch_type\":\"FHIR_R4\"}"'

Sample Response

{
    "batch_statuses": [
        {
            "batch_id": "17610bf3-1883-4594-bd2d-5e794fef0f1e",
            "expected_finish_time": "2024-06-17T06:29:56.000000Z",
            "state": "COMPLETE",
            "query_count": 1090,
            "batch_type": "FHIR_R4"
        },
        {
            "batch_id": "4d85778d-1dc0-4546-92bf-8b7d0f6a12d9",
            "expected_finish_time": "2024-06-17T09:04:57.000000Z",
            "state": "COMPLETE",
            "query_count": 9820,
            "batch_type": "CCDA"
        },
        {
            "batch_id": "5b060a5a-bf07-41dc-ae7d-7bc92003b00c",
            "expected_finish_time": "2023-09-22T12:40:00.000000Z",
            "state": "COMPLETE",
            "query_count": 3,
            "batch_type": "FHIR"
        }
    ]
}

Get Batch

GET https://api.particlehealth.com/api/v1/projects/{project_id}/batches

An in-depth look into the contents and queries inside of a particular batch. Regardless of the batch type you initially created, the response will be the same format.

Sample Request

curl --location 'https://api.scratch.particlehealth.com/api/v1/projects/{{PROJECT_ID}}/batches/{{BATCH_ID}}' \
--header 'Authorization: ••••••'

Sample Response

{
    "batch_id": "{{BATCH_ID}}",
    "expected_finish_time": "2023-09-22T12:40:00.000000Z",
    "state": "COMPLETE",
    "query_count": 3,
    "queries": [
        {
            "query_id": "265dfcbd-fdea-4774-9140-f422a32b7797",
            "state": "COMPLETE",
            "file_count": 21
        },
        {
            "query_id": "5448b742-5894-413d-bd7c-4376e5cd48f9",
            "state": "COMPLETE",
            "file_count": 36
        },
        {
            "query_id": "a05df066-b8a4-4d8d-9b06-36be70acb660",
            "state": "COMPLETE",
            "file_count": 311
        }
    ],
    "batch_type": "FHIR_R4"
}

Updated 6 months ago