Medical imaging pipeline
International internship, Feb–Jun 2026
- My role
- AI & software engineering intern
- Stack
- Python, pydicom, OpenCV, PyTorch, CUDA, OCR, PySide6
- Context
- 2Ai IPCA, Barcelos, Portugal
A Python pipeline that reads the measurements burned into echocardiography studies, finds the video frame each measurement was taken from, and turns the result into structured research data that a person can review.
Problem
Measurements stuck inside images
The measurements the research needed were burned into the echocardiography images instead of being stored as separate fields. Copying them out by hand was slow, hard to repeat, and hard to check across large batches of studies. The researchers also needed to know which ultrasound video each measurement image came from.
Pipeline
OCR, ECG, source matching, and review
- Walk the studies by patient and exam, and load the saved state if a run is resuming.
- Find the measurement panel, cut it into lines, and OCR each line. A word list built from the known labels reranks the OCR guesses, and each line is split into name, value, and unit.
- Extract the ECG trace and its marker dots from the frames.
- For every single-frame measurement image, search the videos of the same exam for the frame it was taken from.
- Write JSON and CSV, give every measurement a status code, and send uncertain rows to a review queue.
Each stage can run as its own pass and resumes where it stopped, so a long batch doesn't start over after an interruption.
Source matching
An image search that took most of the time
Finding the source video is a similarity search. The matcher filters the exam's videos on content time and DICOM metadata, uses the ultrasound region from the metadata as a starting point, and aligns the measurement image against sampled frames with ECC. A coarse pass ranks the videos, a fine pass checks the best ones, and the top three videos come out with their best frame and scores.
This search was the slow part of every run. With OpenCV, findTransformECC took about 96% of the matcher's time, and a profile showed that preparing candidate frames was also expensive. I fixed it in two steps:
- Decode only the videos that pass the filter, decode each one once, and compute each video's mask once. The output stayed identical.
- Port OpenCV's ECC to PyTorch and compile it for CUDA, so it solves the frames of all candidate videos in batches on the GPU. It keeps OpenCV's warp precision, masked statistics, and stopping rules, and falls back to OpenCV when no GPU is available.
On a benchmark of 39 measurement images, source matching went from 1182 seconds to 234 seconds with the faster frame preparation, and to 80 seconds with the GPU ECC and three workers. That's 14.9 times faster. On 192 measurements, the GPU version picked the same best video as OpenCV every time and the same best frame 186 times. OpenCV matches its own result 184 times when its input is nudged by one floating-point step, so the GPU version is as close to OpenCV as OpenCV is to itself.
Engineering
Reliable enough for research work
- Resumable workSaved state means an interrupted batch doesn't start over, and each stage can be rerun on its own.
- Human validationEvery measurement gets a status code from 0 to 4. A PySide6 review tool picks rows by status so a person can accept or correct them.
- Speedups that keep the resultsI rejected faster settings, like fewer ECC iterations or warm starts from nearby frames, when they changed the rankings.
- Traceable outputThe exports are keyed by patient, exam, and DICOM, so every value points back to its source study.