Tenzaa's Tech Blog
This site is dedicated to sharing IT-related technologies.
  Post Categories
Data Engineering 27 Architecture 25 Web Architecture 2 Observability 24 MLOps 16 Cloud Native 11 Distributed Systems 28 Software Architecture 4 MLOps Engineering 1 Backend Architecture 2 Backend Development 8 DevOps 21 System Design 6 System Architecture 4 DevSecOps 4 Mobile Development 2 Security 3 Cloud Security 2 Full-Stack Development 1 Cloud Architecture 3 Security Engineering 1 Platform Engineering 1 WebAssembly 1 Serverless 1 Software Engineering & Architecture 1 Real-Time Communication 1
Building a Stateful CDC Processor with Micronaut and XState for Apache Hudi Upserts from MongoDB Building a Stateful CDC Processor with Micronaut and XState for Apache Hudi Upserts from MongoDB
The standard approach to Change Data Capture (CDC) pipelines often results in a messy, difficult-to-query data lake. Sim
2023-10-27 Data Engineering
Apache Hudi CDC Kafka MongoDB XState Micronaut
Implementing an Exactly-Once CDC Ingestion Pipeline with RocketMQ Transactional Messages and Idempotent NoSQL Writes Implementing an Exactly-Once CDC Ingestion Pipeline with RocketMQ Transactional Messages and Idempotent NoSQL Writes
The initial data lake ingestion pipeline was hemorrhaging data integrity. Our mandate was simple: every database change
2023-10-27 Data Engineering
CDC Data Lake Distributed & Middleware RocketMQ NoSQL MongoDB
Implementing Dynamic JWT-Driven Row and Column Security for a GraphQL Data Lake API Implementing Dynamic JWT-Driven Row and Column Security for a GraphQL Data Lake API
The central problem was an analytics API spiraling out of control. Our multi-tenant platform dumps billions of events in
2023-10-27 Data Engineering
Security JWT Data Lake GraphQL Trino Iceberg
Implementing a Resilient Time Series Aggregation System Using Dask with an etcd-based Control Plane Implementing a Resilient Time Series Aggregation System Using Dask with an etcd-based Control Plane
The initial deployment of our time series aggregation pipeline seemed straightforward. A Dask cluster was spun up, parti
2023-10-27 Data Engineering
Distributed Systems Python etcd Dask Time Series Fault Tolerance
Implementing a Transaction-Aware Data Synchronization Pipeline from a Relational Database to Delta Lake for FastAPI Endpoints Implementing a Transaction-Aware Data Synchronization Pipeline from a Relational Database to Delta Lake for FastAPI Endpoints
The mandate was clear: provide near real-time analytical access to our core operational data without compromising the pe
2023-10-27 Data Engineering
Software Engineering & Architecture Delta Lake MyBatis FastAPI Database
Building Immutable Ingestion Infrastructure on AKS for React Native Analytics with Hudi, Packer, and Jest Building Immutable Ingestion Infrastructure on AKS for React Native Analytics with Hudi, Packer, and Jest
Our mobile analytics ingestion pipeline was a constant source of on-call alerts. It was a collection of brittle scripts
2023-10-27 Data Engineering
Jest Azure AKS Apache Hudi Packer React Native
2 / 5