Life of a Query

When the Particle Health platform receives a query request to retrieve clinical records for a set of patient demographics, we serve back a response with an indication that we're doing some processing behind the scenes.

But, what exactly is going on under the hood? We've outlined the life of a query below so that you can better understand how the platform works.

1. Update the Master Patient Index: Update Particle's Master Patient Index to ensure we always have the latest demographics being represented for a given patient.

2. Record Locator: Use the geographic information provided in the patient's demographics, as well as any location hints provided, to feed our proprietary Record Locator algorithm to determine all of the network partners that may have data for this request.

3. Subquery Split: Using this list of network partners, we split the query into a multitude of subqueries, each representing the patient demographics and a unique partner endpoint to request data from. Queries are typically split into hundreds of subqueries during this process.

4. Query Networks: Using the partner endpoint provided, each subquery makes requests for information about the patient demographics. While each network partner has its own process that happens behind the scenes, it always boils down to the same three network calls for each. They first identify if they have data for the patient based on the demographics provided. Then they identify if they have any associated documents, and finally, they will return all documents. Please see Network Variability for some of the issues that may come up during this step.

5. Record Validator: Once Particle has retrieved the queried patient's documents from the networks, we compare the patient demographics provided by the customer against the patient demographics included in the retrieved C-CDA documents to minimize [even the low] likelihood of Particle's customers receiving incorrect patient records from the EMRs. Further details can be found here.

6. Data Transformation: Subqueries are responsible for retrieving C-CDA files in XML format directly from the EMRs that Particle is connected to. Once all subqueries have been completed, we transform the XML files into the customer's preferred data format e.g. FHIR R4, Flat, or other. Non-XML files retrieved from the networks (e.g. PDFs, PNGs, etc.) are not converted, but can still be retrieved via our C-CDA endpoints. Further details about the data transformation processes can be found here.

7. Data Aggregation: With hundreds of subqueries all making requests in parallel, it's a tricky process to ensure we've retrieved all of the data before wrapping up processing. As such, every time a subquery completes, we run checks to determine the number of subqueries still outstanding. When that number hits zero, we take all of the resulting C-CDA files and bundle them into a single ZIP file, and make them available in the appropriate data stores (depending on the customer's preferred data format) for retrieval. The query is then marked complete.

8. Data Made Available: Once the query is complete, the data can be retrieved based on the API that was used to initiate the process.

   1. C-CDA queries can retrieve the C-CDA files or bundled ZIP file from the /Files/ endpoint.
   2. FHIR queries can retrieve FHIR R4 JSON data using the /R4/ endpoint.
   3. Flat queries can be retrieved via the flat/{id}/collect-data endpoint.