LLM infrastructure

AI Core

The LLM stack, layer by layer.

Tokenization with BPE, semantic chunking, multimodal document parsing and an annotation studio for RLHF/DPO data; contrastive embeddings, an HNSW vector database, knowledge-graph extraction, hybrid dense + BM25 retrieval and cross-encoder reranking.

On top: an inference server with token streaming and dynamic batching, a semantic model router, bidirectional guardrails, ReAct agents with structured memory and a sandboxed tool runtime — and the MLOps loop around them: synthetic data, dataset versioning, QLoRA fine-tuning, an eval harness and tracing. A hub orchestrates all of it.

Repositories: 22
Lines: 25,716
Commits: 255
Test files: 43
Built: Jun 2026 – Jul 2026
Mostly: Python

↳ Flagship

Where it all comes together

ai-core-infra-hubInfra HubA centralized, interactive web control panel and showcase hub integrating and orchestrating all 21 AI Core infrastructure projects.14.3k loc · 42 commits · 22 test files nexus-second-brainNexusA privacy-first personal knowledge management SPA integrating semantic search, document parsers and chat.1.2k loc · 11 commits · 1 test files

Data & input

Doc Parsermultimodal-doc-parserA document parsing pipeline that extracts structured Markdown, tables and visual elements from PDFs and images.402 loc9 commits1 tests BPE Tokenizerbpe-tokenizer-from-scratchA lightweight Byte Pair Encoding (BPE) tokenizer built completely from scratch for LLM data pipelines.358 loc9 commits1 tests Semantic Chunkersemantic-chunking-engineAn embedding-based document chunker that segments text at semantic transitions for RAG.511 loc10 commits1 tests Annotation Studiollm-annotation-studioA web-based annotation studio for text classification, RLHF/DPO alignment data and NER tagging.1.7k loc9 commits1 tests

Representation & retrieval

Knowledge Graphknowledge-graph-extractorAn extraction pipeline that builds structured knowledge graphs from unstructured text using LLMs.918 loc10 commits1 tests Embedding Trainercontrastive-embedding-trainerContrastive training of text embeddings with PyTorch and Siamese networks using a triplet cosine loss.540 loc9 commits1 tests Vector DBnano-vector-dbA lightweight vector database built from scratch implementing HNSW indexing and cosine similarity search.784 loc9 commits1 tests Hybrid Retrievalhybrid-search-retrieval-pipelineA hybrid retrieval engine combining dense vector search and sparse BM25 keyword matching.629 loc16 commits1 tests Cross-Encodercross-encoder-rerankerA high-precision cross-encoder pipeline to re-rank and filter retrieved documents for RAG.341 loc12 commits1 tests

Inference & control

Guardrailsllm-guardrails-shieldA bidirectional security firewall protecting LLMs against prompt injection and data leaks.225 loc12 commits1 tests Model Routersemantic-model-routerA semantic router that assigns each prompt to the optimal LLM to reduce latency and API cost.257 loc9 commits1 tests Inference Serverllm-inference-serverA high-performance LLM inference server with token streaming and dynamic request batching.488 loc11 commits1 tests

Agents

Agent Memoryagentic-memory-layerA structured memory layer providing short-term and long-term vector memory for AI agents.302 loc9 commits1 tests ReAct Agentsorchestra-agentsA framework for multi-agent collaboration using autonomous ReAct reasoning loops.380 loc9 commits1 tests Tool Sandboxsecure-tool-runtimeA secure, sandboxed execution runtime that lets LLMs run code and call APIs safely.254 loc9 commits1 tests

MLOps & quality

Synthetic Datasynthetic-data-generatorAn automated engine to generate and filter high-quality synthetic instruction datasets using LLMs.260 loc9 commits1 tests Dataset VCSdataset-version-controlA version control utility for machine learning datasets to guarantee reproducible training runs.339 loc11 commits1 tests QLoRA Finetunerllm-qlora-finetunerAn efficient LLM fine-tuning pipeline using QLoRA and PEFT on custom datasets with PyTorch.489 loc10 commits1 tests Eval Harnessllm-eval-harnessAn automated evaluation suite to benchmark LLM generation quality and detect regressions.439 loc10 commits1 tests Tracerllm-observability-tracerEnd-to-end monitoring and tracing for LLM API calls, token counts and request latency.582 loc10 commits1 tests