Building Private, Offline-Ready AI Experiences

[juliamuiruri4]

Welcome to the Local AI discussion hub! 🎉

This is a dedicated space for our community to collaborate, learn, and share everything related to building AI applications that run entirely on-device.

What is Foundry Local?

Foundry Local enables you to run AI models and integrate them into your applications directly on your device, with privacy by design, zero cloud dependency, and fast inference. Whether building for the JavaScript AI Buildathon or exploring local AI in general, this is your space.

Use This Discussion For:

• 💬 Questions & Troubleshooting: Running into issues with the quest, setup, model downloads or inference? Ask away!
• 🏗️ Architecture & Best Practices: Discuss design patterns, optimization strategies and how to structure offline-first applications.
• 📚 Learning & Resources – Share tutorials, documentation links, and helpful examples.

🏆 JavaScript AI Build-a-thon Hack!

As part of the Build-a-thon Hack!, we have a special award category that will recognize the best performing AI solution with standout offline capabilities (local inference).

Getting Started:

New to Foundry Local? Start here:

• Local AI Development with Foundry Local Livestream
• Local AI Development Quest: Comprehensive walkthrough of core concepts
• Foundry Local (ALL resources)

Community Guidelines:

• Be respectful and inclusive
• Search existing threads before posting duplicates
• Share code snippets and error logs when asking for help
• Help others! Your expertise matters

Have a question, project or insight to share? Start a new reply or thread below!

Discussion host: @HamidOna

[HamidOna]

Excited to be here! Good luck to everyone participating

[BY3D]

Hello, when an AI model is downloaded via Microsoft's Foundry CLI or through VS Code's AI Toolkit extension, is that model running exclusively on our computers offline?

For example, I downloaded the phi-4-mini model and asked it about it being offline, it responded by saying that it processes its results in the cloud. Am I correct in assuming that this particular model is unaware about it being run locally/offline on my computer?

[juliamuiruri4]

@BY3D - is that model running exclusively on our computers offline? - YES, once you pull/ download the model and start the foundry local service, it processes your requests locally on-device without sending any data to the cloud

[BY3D]

Thanks for the reply. Now it make sense why phi-4-mini was replying to me instantaneously.

[juliamuiruri4]

Foundry Local Livestream Recap & Summary

Thanks to everyone who joined yesterday's insightful sessions on Foundry Local and local AI development!

• Rewatch the Livestream
• Complete the Foundry Local quest
• Resources:
  • https://github.com/microsoft/olive - Model optimization toolkit
  • https://github.com/microsoft/olive-recipes

Here's a summary of the key questions and topics discussed. Please continue the conversations below ...

Foundry Local vs. Ollama

Is Foundry Local essentially Microsoft's version of Ollama?
When should you choose Foundry Local over Ollama?

• Foundry Local is self-contained with no external dependencies required for end users, while Ollama requires separate installation. For enterprise app deployment, Foundry Local offers a smoother experience.
• CLI/Exploration: For model exploration and capabilities, both are very similar.
• Integration: Ollama currently integrates more easily with tools like OpenClaw, but Foundry Local is rapidly expanding integration support.

System Requirements & Platform Support

Please mention about system configuration requirements to work on Foundry locally
Assuming/hoping that DLL is just one option and that we can run locally on MacOS as well (not Windows-only)?

• Not Windows-only! Foundry Local runs on MacOS and other platforms.
• System requiremetns to run Foundry Local

Azure Subscription & Offline Capabilities

Do we need any azure sub? or we can download and apps can work offline/onprem?

• No Azure subscription required. Foundry Local enables fully offline, on-premises AI workflows.

Model availability & Integrations

Does Foundry local support latest models like Qwen3.5?
How does Foundry Local compare to WebLLM for in-browser inference?
I've tried using Github Copilot using Ollama and Cline on a 5090 GPU. Not much luck. Want to pivot to Foundry Local and see if it is smoother.

• Curently Not available, but coming soon
• FL Models
• GitHub Copilot can now use Foundry Local models, keeping AI coding assistance entirely on-device
• FL runs as local service with OpenAI-compatible API on localhost, browser apps can call via fetch() after setup, but not pure in-browser like WebLLM
• Recently added support for tool calling and structured outputs to Foundry Local v2 SDK

Happy building!