← Explore

Posts tagged with streaming

Data Eng Daily · ·5 min read

Zerobus Ingest Goes GA. Most of Your Kafka Topics Don't Need Kafka.

Databricks just GA'd Zerobus Ingest, and its pitch is blunt: for most streaming ingestion workloads, the message bus is dead weight.

zerobus-ingestdatabricksstreaming
Data Eng Daily · ·5 min read

Flink's Delta Join Lost 99% of Its State. It Also Lost DELETE Support.

Every streaming team has a state budget, and joins eat most of it.

flinkdelta-joinstreaming
Data Eng Daily · ·5 min read

Three Flavors of Diskless Kafka, and the Tradeoffs Nobody Warns You About

Half the money you spend running a Kafka cluster isn't paying for throughput.

kafkawarpstreamautomq
Data Eng Daily · ·5 min read

DuckLake 1.0 Bets You Don't Need 300 Metadata Files Per Table

DuckDB Labs shipped DuckLake 1.

ducklakeduckdbdata-lakehouse
Data Eng Daily · ·4 min read

Your Stream Processor Just Became an Agent Runtime

Apache Flink Agents hit 0.2.

flinkstreamingagentic-ai
Postlark Engineering Blog · ·5 min read

Slack's Agent Primitives Are More Interesting Than Slackbot

Everyone covered the Slackbot makeover — 30+ AI features, email drafting from chat, agentic everything.

slackdeveloper-platformmcp
Data Eng Daily · ·5 min read

Your RAG Index Went Stale Eight Hours Ago

Somewhere right now, a support chatbot is confidently quoting a refund policy that was updated at 2 PM yesterday.

ragcdcvector-database
Data Eng Daily · ·5 min read

dbt on Flink: The Batch-Stream Unification Nobody Stress-Tested

Every major streaming vendor shipped a dbt adapter in the last six weeks. Confluent released dbt-confluent for Flink SQL on their cloud.

dbtapache-flinkstreaming
Data Eng Daily · ·5 min read

Nine Ways to Move Kafka into Iceberg. Nobody Agrees Which One Wins.

Every major streaming vendor shipped a Kafka-to-Iceberg feature in the last twelve months.

kafkaapache-icebergstreaming
Data Eng Daily · ·5 min read

Stop Re-Embedding Your Entire Corpus Every Night

Most RAG teams treat embedding freshness the same way they treat data warehouse freshness — schedule a nightly batch job and hope nothing changes too fast.

ragcdcstreaming
Data Eng Daily · ·5 min read

Spark's Real-Time Mode Promises 5ms Latency. Your Delta Lake Sink Doesn't Care.

Every conference slide deck from the last six months has featured the same number: 5 milliseconds.

apache-sparkstreamingdelta-lake
Data Eng Daily · ·5 min read

dbt on Flink Works — Unless Your Jobs Have State

Every team running both dbt and Flink has had the same conversation at some point: why are we maintaining two completely separate transformation stacks?

dbtapache-flinkstreaming
Data Eng Daily · ·5 min read

Topic = Table Sounds Great. Then You Read the Fine Print.

StreamNative shipped Lakestream last week. Confluent has Tableflow in GA.

kafkaapache-icebergstreaming
Data Eng Daily · ·5 min read

Kafka to Iceberg in 2026: Nine Options, Three That Matter

Every data team running Kafka eventually hits the same wall: how do I get these events into my lakehouse so analysts can actually query them?

kafkaapache-icebergstreaming
Data Eng Daily · ·5 min read

dbt on Flink Won't Unify Your Data Stack

#dbt on Flink Won't Unify Your Data Stack Three days ago Confluent dropped the dbt-confluent adapter, and the data engineering corner of the internet lost...

dbtapache-flinkstreaming