A web application that converts any image into an interactive 3D scene using Apple's SHARP model for Gaussian splatting. https://sharp-ml.vercel.app/
- Modal Account — Sign up at modal.com
- Vercel Account — For deploying the web app
- Google Cloud Console — For OAuth credentials
- PostgreSQL Database — Use Neon, Supabase, or Railway
- Python 3.10+ — For deploying the Modal endpoint
# Install Modal CLI
pip install modal
# Authenticate with Modal
modal token new
# Navigate to the modal directory and deploy
cd modal
modal deploy sharp_api.pyAfter deployment, Modal will display your endpoint URL. It looks like:
https://YOUR_USERNAME--apple-sharp-sharpmodel-generate.modal.run
Copy this URL — you'll need it in the next step.
Create a PostgreSQL database and run migrations:
# Install dependencies
npm install
# Run Prisma migrations
npx prisma migrate deploy- Go to Google Cloud Console
- Create a new project or select existing
- Enable the Google+ API
- Go to Credentials → Create Credentials → OAuth 2.0 Client ID
- Add authorized redirect URI:
https://your-domain.com/api/auth/callback/google - Copy the Client ID and Client Secret
Or deploy manually:
# Install Vercel CLI
npm i -g vercel
# Deploy
vercelIn your Vercel project settings, add all required environment variables:
# Database
DATABASE_URL="postgresql://..."
# NextAuth.js
NEXTAUTH_URL="https://your-domain.com"
NEXTAUTH_SECRET="generate-with-openssl-rand-base64-32"
# Google OAuth
GOOGLE_CLIENT_ID="..."
GOOGLE_CLIENT_SECRET="..."
# App
NEXT_PUBLIC_APP_URL="https://your-domain.com"
# Modal
MODAL_ENDPOINT_URL="https://..."
# Vercel Blob (auto-added when you enable Blob storage)
BLOB_READ_WRITE_TOKEN="..."- Go to your project on Vercel
- Navigate to Storage → Create Database → Blob
- This automatically adds the
BLOB_READ_WRITE_TOKEN
npm installCreate a .env.local file with all the variables from the .env.example file.
# Generate Prisma client
npx prisma generate
# Run migrations (or push for development)
npx prisma db pushnpm run devOpen http://localhost:3000 in your browser.
- Sign In — Use Google OAuth to create an account
- Upload an Image — Drag and drop or click to select a photo (PNG, JPG, or WebP)
- Wait for Processing — The image is processed via Modal (~1-10 seconds, longer on cold start)
- Explore in 3D — Use your mouse to navigate the 3D Gaussian splat scene:
- Left-click + drag — Rotate the view
- Scroll wheel — Zoom in/out
- Right-click + drag — Pan the view
- Share — Copy the shareable link to share your 3D scene
- Upgrade — After 3 free scenes, upgrade via Stripe for unlimited access
├── modal/
│ └── sharp_api.py # Modal deployment for Apple Sharp
├── prisma/
│ └── schema.prisma # Database schema (users, accounts)
├── app/
│ ├── components/
│ │ ├── AuthGate.tsx # Authentication gate
│ │ ├── UpgradeModal.tsx # Stripe payment modal
│ │ ├── ImageUpload.tsx # Drag & drop upload zone
│ │ ├── GaussianViewer.tsx # 3D model viewer (PLY Gaussian splats)
│ │ └── ProcessingStatus.tsx # Progress indicator
│ ├── api/
│ │ ├── auth/[...nextauth]/route.ts # NextAuth handler
│ │ ├── user/route.ts # User data API
│ │ ├── stripe/
│ │ │ ├── checkout/route.ts # Create checkout session
│ │ │ └── webhook/route.ts # Handle Stripe webhooks
│ │ └── process/route.ts # Image processing (authenticated)
│ ├── providers.tsx # Session provider wrapper
│ ├── page.tsx # Main application page
│ └── globals.css # Custom styling
├── lib/
│ ├── auth.ts # NextAuth configuration
│ ├── prisma.ts # Prisma client
│ └── stripe.ts # Stripe client
- Framework — Next.js 16 with App Router
- Authentication — NextAuth.js v5 with Google OAuth
- Database — Prisma with PostgreSQL
- Styling — Tailwind CSS 4
- Animations — Framer Motion
- 3D Rendering — Three.js + @mkkellogg/gaussian-splats-3d
- Icons — Lucide React
- ML Inference — Modal (serverless GPU)
- File Storage — Vercel Blob
| Variable | Required | Description |
|---|---|---|
DATABASE_URL |
Yes | PostgreSQL connection string |
NEXTAUTH_URL |
Yes | Your app URL |
NEXTAUTH_SECRET |
Yes | Random secret for JWT signing |
GOOGLE_CLIENT_ID |
Yes | Google OAuth client ID |
GOOGLE_CLIENT_SECRET |
Yes | Google OAuth client secret |
NEXT_PUBLIC_APP_URL |
Yes | Your app URL (for redirects) |
MODAL_ENDPOINT_URL |
Yes | Your Modal Sharp endpoint URL |
BLOB_READ_WRITE_TOKEN |
Yes | Auto-provided by Vercel Blob |
- Free tier: 10 scene generations per user
The modal/sharp_api.py file configures:
- GPU: A10G (good performance/cost ratio)
- Timeout: 300 seconds
- Container idle: 300 seconds (keeps warm for faster subsequent requests)
- Model caching: Uses Modal Volume to cache the 1.4GB model weights
- Apple SHARP on Hugging Face
- Apple SHARP GitHub
- SHARP Project Page
- SHARP arXiv Paper
- Modal Documentation
The Modal container is starting up. This takes 30-60 seconds on cold start. Try again after waiting.
Check that:
- Your
MODAL_ENDPOINT_URLis correct - The Modal app is deployed (
modal deploy sharp_api.py) - Your Modal account is active
Make sure you're signed in with Google. The app requires authentication to protect against abuse.
The first request after inactivity requires:
- Starting the Modal container (~10-20s)
- Loading the Sharp model (~10-20s)
- Running inference (~1-5s)
Subsequent requests while the container is warm are much faster (~1-5s).
This web interface is open source under the MIT License.