The Life of a Query

When the Particle Health platform gets a query request to retrieve clinical records for a set of patient demographics, we serve back a response with an indication that we're doing some processing behind the scenes. But what exactly is going on under the hood? We've outlined the life of a query below so that you can better understand how the platform works.

1. Update the Master Patient Index: The first step each query takes is updating Particle's Master Patient Index so we can ensure we always have the latest demographics being represented for a given patient.

2. Record Locator: Next, we'll use the geographic information provided in the patient's demographics, as well as any location hints provided, to feed our proprietary Record Locator algorithm to determine all of the network partners that may have data for this request.

3. Subquery Split: Using this list of network partners, we split the query into a multitude of subqueries, each representing the patient demographics and a unique partner endpoint to request data from. Queries are typically split into hundreds of subqueries during this process.

4. Query Networks: Using the partner endpoint provided, each subquery makes requests for information about the patient demographics. While each network partner has its own process that happens behind the scenes, it always boils down to the same three network calls for each. They first identify if they have data for the patient based on the demographics provided, then they identify if they have any associated documents, and finally, they will return all documents. Also see Network Variability for some of the issues that may come up.

5. Data Transformation: Subqueries are responsible for retrieving C-CDA files in XML format directly from the EMRs we have connectivity to. Once all subqueries have been completed, we transform the XML files into FHIR R4 and our Flat format. Non-XML files retrieved from the networks (e.g. PDFs, PNGs, etc.) are not converted, but can still be retrieved via our C-CDA endpoints.

6. Data Aggregation: With hundreds of subqueries all making requests in parallel, it's a tricky process to ensure we've got all of the data back before wrapping up processing. Every time a subquery completes, we run checks to determine the number of subqueries still outstanding. When that number hits zero, we take all of the resulting C-CDA files and bundle them into a single ZIP file, and our FHIR and Flat data stores for retrieval. The query is then marked complete.

7. Data Made Available: Once the query is complete, the data can be retrieved based on the API that was used to initiate the process.

   1. C-CDA queries can retrieve the C-CDA files or bundled ZIP file from the /Files/ endpoint.
   2. FHIR queries can retrieve FHIR R4 JSON data using the /R4/ endpoint.
   3. Flat queries can be retrieved via the flat/{id}/collect-data endpoint.