Structured Data RAG Engineer

Our client, an international AI development company based in New York, is currently seeking a Structured Data RAG Engineer to lead the design and development of Retrieval-Augmented  Generation (RAG) systems optimized for structured data sources such as relational databases, APIs, and enterprise  data warehouses. 

The ideal candidate will have experience using advanced frameworks like GraphRAG and  LangGraph to enable reasoning over structured, interconnected datasets. This role bridges data engineering with  LLM capabilities, delivering natural language interfaces and intelligent querying systems for enterprise data. 

Key Responsibilities 

Structured Data Integration: 

Develop connectors and pipelines to ingest structured data from SQL/NoSQL databases, APIs, and data  lakes 

Convert tabular and relational data into vectorized and graph-structured formats for LLM access 

RAG System Development: 

Build structured data RAG systems using GraphRAG and LangGraph to enable multi-hop reasoning and  context-aware retrieval 

Integrate vector databases (e.g., FAISS, Milvus, Weaviate) for scalable retrieval over structured corpora 

LLM Alignment & Prompt Engineering: 

Design prompt templates tailored to structured queries and logic-driven reasoning 

Enhance accuracy and interpretability of LLM outputs in structured data environments

Model Evaluation & Optimization: 

Write custom evaluation code and design benchmarking frameworks 

Measure response accuracy, relevance, latency, and scalability across large data sets 

Deployment & Automation: 

Leverage cloud infrastructure (GCP, AWS, or Azure) and implement CI/CD pipelines 

Collaborate with ML, data, and backend engineering teams to productionize RAG systems

Qualifications & Skills

Expertise in Retrieval-Augmented Generation (RAG), including structured/tabular data use cases 

Strong proficiency with GraphRAG approaches and multi-agent development using LangGraph 

Experience with graph databases, including LightRAG or Microsoft Graph RAG 

Solid background in SQL, Redshift, relational databases, and structured APIs 

Proficient in Python, LangChain, vector stores, and agentic orchestration 

Familiarity with LLMs, prompt engineering, and structured context integration 

Strong analytical mindset and attention to detail 

Collaborative and communicative across cross-functional teams 

Proactive, innovative thinker with a product-oriented approach to AI applications

Very strong English communication skills, both written and verbal (essential for global collaboration)