Skip to content

Hands-on Tutorial

Welcome to the concierge hands-on tutorial. This guide walks you through the current application step by step.

Each step explains the why behind the change and shows runnable commands plus selected code excerpts.

Why follow this tutorial?

The repository is a template for building LLM applications on top of Microsoft Foundry with LangChain and LangGraph. Rather than reading the finished code in isolation, you will build it up step by step so the design decisions stay visible.

The fastest path through the tutorial is top to bottom. Each step depends only on what comes before it, and each one is independently verifiable.

Step Topic What it adds Needs Azure?
1 Microsoft Foundry + LangChain A Typer CLI that calls a Foundry-hosted model yes
2 Observability (Tracing & MLflow) Trace LangChain runs to Azure Monitor and to a local MLflow UI yes / partly1
3 PostgreSQL (pgvector) CRUD A persistent vector store, locally or on Azure optional2
4 LangGraph Todo Agent CLI A ReAct agent that operates the Todo Web API through tools yes
5 MLflow GenAI Evaluation Evaluate and compare agent output quality with heuristic, LLM-judge, and custom scorers partly3

Already exploring locally?

The Todo App (Clean Architecture) section runs without any Azure credentials and is the best entry point if you just want to see the FastAPI / Typer / repository layout in action.

High-level architecture

flowchart LR
    User([Developer])
    CLI["Typer CLI<br/>scripts/microsoft_foundry/vanilla.py"]
    Settings["Pydantic settings<br/>concierge/settings/*"]
    LC["LangChain / LangGraph"]
    Foundry[("Microsoft Foundry<br/>Project endpoint")]
    Models["Foundry-hosted models<br/>gpt-5, text-embedding-3-small, ..."]
    Tracer["AzureAIOpenTelemetryTracer"]
    Monitor[("Azure Monitor")]
    MLflow[("MLflow Tracking<br/>http://127.0.0.1:5000")]

    User --> CLI
    CLI --> Settings
    CLI --> LC
    LC -->|"chat / embeddings / agent"| Foundry
    Foundry --> Models
    LC -.->|"--tracing"| Tracer --> Monitor
    LC -.->|"--mlflow autolog"| MLflow

Prerequisites

Before starting, make sure your machine has the following installed. Versions align with what the repository's pyproject.toml and Makefile expect.

  • Python 3.10+
  • uv - dependency and virtual-env manager used by every make target
  • GNU Make - thin wrapper around the uv commands
  • An Azure subscription with access to Microsoft Foundry and deployed chat / embedding models. The examples use the default deployment names from the code (gpt-5 and text-embedding-3-small), but you should replace them with the deployment names available in your project.
  • Azure CLI signed in (az login) so DefaultAzureCredential can pick up your identity

Why DefaultAzureCredential?

The CLI in this repo authenticates via DefaultAzureCredential, which transparently uses az login, managed identity, environment variables, or a developer credential. You do not need to manage API keys.

How to read each step

Every step page is structured the same way so you can pattern-match quickly:

  1. Goal - the user value the step unlocks.
  2. Why - the design rationale.
  3. Steps - runnable commands and selected code excerpts.
  4. Verify - how to confirm the change works.
  5. Troubleshooting - common pitfalls and fixes.

Continue with Step 1 - Microsoft Foundry + LangChain.

  1. MLflow runs entirely on your laptop; only the Azure Monitor side requires Foundry tracing to be enabled. ↩

  2. Step 3 has a --fake-embeddings flag that bypasses Foundry entirely, and the --target docker default uses local Docker Compose pgvector. ↩

  3. Steps 5a–5c and 5e run fully on your laptop; only the judge subcommand (Step 5d) requires Azure credentials and a deployed chat model. ↩