The problem
A digital twin is only as useful as the data flowing into it. The platform took live streams from instrumented equipment, ran them through simulation and analytics, and presented the results to researchers who needed to trust what they were looking at.
The hard part was never the simulation. It was everything around it: incoming data that did not always match its own documentation, several groups of users who should not all see the same things, and researchers who needed to reason about the system without reading Python.
Constraints
- Inputs were untrusted. Data arrived from multiple sources with inconsistent shapes, missing fields, and occasional nonsense values. A single bad record reaching the analytics layer produced output that looked plausible and was wrong, which is worse than an error.
- Access had to be granular. Different research groups, different permissions, and an audit trail that had to hold up.
- The audience was not engineers. The people relying on the output were domain researchers. Anything they could not verify themselves became a question in my inbox.
What I built
Validation at the boundary
Every ingest path got a Pydantic model. Not a loose one: required fields required, ranges bounded, enums closed. Anything that failed was rejected at the edge with an error that named the field and said what was wrong, rather than being coerced into something the pipeline would accept.
This is the decision I would repeat on every project. It moves data problems from the hardest place to debug, which is deep inside a transformation, to the easiest, which is the request that caused them.
One permission model, enforced twice
JWT authentication with role-based access control, applied on the API and mirrored in the React interface. The API is the enforcement point. The job of the UI is only to avoid showing people doors they cannot open. Protected routes, token handling, session management, and a CORS policy that named origins explicitly rather than reflecting whatever asked.
Audit logging on the actions that mattered, so that when someone asked who changed a configuration, the answer was a query rather than an investigation.
Documentation as part of the contract
Every endpoint documented in Swagger and OpenAPI. The front end was built against that contract, and I verified front-end behaviour against it before release rather than after a bug report. When a researcher asked what an endpoint returned, I sent them a URL.
The environment around it
AWS across EC2, S3, API Gateway and IAM, with CI/CD through Docker, Git, and Jenkins. Containers meant the environment that ran the tests was the environment that ran in production, which quietly removed a whole category of issue.
What changed
I introduced systematic peer code review and a shared set of engineering standards, alongside PyTest coverage on the logic that carried real risk. Defects dropped. Not because review catches every bug, but because knowing someone will read your code changes what you write.
What I would do differently
I would invest in contract testing between the API and the front end earlier. Manual verification against the OpenAPI spec worked, and it caught real mismatches, but it depended on me remembering to do it. A generated client, or a test suite that fails when a response shape drifts, would have made that guarantee structural instead of behavioural.