Thanks for kicking off the next round - we're now here, right?

we're stacking quite some integration concepts onto each other from architecture point of view - it's not easy to grasp the difference from Ingester and Indexer - especially for me that term is connected to ETL/ELT, but load and transform are part of Indexer. but that's me maybe.

and we're keeping flaws like that sources state in the Ingester now.

compared to

I missed to catch up on your arguments in #997, but will create a sketch PR tho.

wahoo, the diagrams are cool 👏🏼

t's not easy to grasp the difference from Ingester and Indexer - especially for me that term is connected to ETL/ELT, but load and transform are part of Indexer. but that's me maybe.

I don't know, for me it's clear. But I'm biased since I partially built it 😁
I could rename Ingester to Etl to make it clearer

Anyway, we must have something to index (fileter, transform, vectorize, store) documents. I don't need the loader thing, since data are coming in real time from RabbitMQ.