01 Zakres zadań
- Design, build and run end-to-end pipelines covering ingestion, extraction, transformation, enrichment, validation and publication
- Develop reusable connectors and processing components for structured and unstructured sources (APIs and other interfaces)
- Build reliable pipelines with idempotent writes, deduplication, retries, replay, backfills and recovery from partial failures
- Design incremental and full-load patterns suited to each source’s volume and change behaviour
- Develop SQL data models and transformations, with a focus on integrity and query performance
- Design graph schemas and translate ontology concepts into implementable models, mapping source data to the knowledge graph
- Build graph loading and query patterns that preserve identifiers, relationships, provenance and lineage
- Design NoSQL models around access patterns, and integrate relational, graph and NoSQL stores
