Skip to content

High-performance open-source synthetic data engine. Uses LLMs for schema design and vectorized NumPy for deterministic, scalable generation.

License

Notifications You must be signed in to change notification settings

rasinmuhammed/misata

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

20 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

🧠 Misata

The Intelligent Synthetic Data Engine

PyPI version Python versions License Downloads

Stop writing fake data scripts.
Generate production-grade datasets from natural language.

Quick Start β€’ Features β€’ Python API β€’ Enterprise


πŸš€ Why Misata?

Misata isn't just a random data generator. It's an intelligent engine that understands your business logic, relationships, and constraints. Whether you need 50 rows for unit tests or 10 million rows for load testing, Misata delivers statistically realistic data that looks and behaves like the real thing.

Feature Faker SDV Misata
Natural Language Input ❌ ❌ βœ…
Auto Schema Generation ❌ ❌ βœ…
Relational Integrity ❌ βœ… βœ…
Business Constraints ❌ ❌ βœ…
No Training Data Needed βœ… ❌ βœ…
Streaming (10M+ rows) ❌ ❌ βœ…

⚑ Quick Start

1. Install

pip install misata

2. Generate

Describe what you need in plain English. Misata handles the rest.

# Basic generation (Rule-based, instant)
misata generate --story "A SaaS platform with 50K users, monthly subscriptions, and a 20% churn rate in Q3"

# Intelligent generation (LLM-powered)
export GROQ_API_KEY=gsk_...
misata generate --story "E-commerce store with seasonal trends and customer segments" --use-llm

3. Result

Misata creates a relational schema, generates the data, and saves it to ./generated_data.

πŸ“‹ Schema: SaaS_Platform
   Tables: 4 (users, subscriptions, payments, events)
   Relationships: 3
   Events: 1 (Churn Spike Q3)

πŸš€ Performance: 385,000 rows/second
πŸ’Ύ Data saved to: ./generated_data

πŸ”₯ New in v0.5.0

πŸ”„ Schema Introspection & Seeding

Already have a database? Misata can reverse-engineer your schema and seed it with realistic data.

# 1. Introspect your existing DB
misata schema --db-url postgresql://user:pass@localhost:5432/mydb --output schema.yaml

# 2. Seed it with 100K rows of realistic data
misata generate --config schema.yaml --db-url postgresql://... --db-truncate

πŸ“ˆ Reverse Engineering from Charts

Describe a chart, and Misata generates the underlying data to match it.

misata graph "Monthly revenue growing from $10k to $1M over 2 years, with a dip in August"

πŸ’» Python API

Seamlessly integrate Misata into your test suites and CI/CD pipelines.

Standard Generation

from misata import DataSimulator
from misata.llm_parser import LLMSchemaGenerator

# 1. Design schema with AI
llm = LLMSchemaGenerator(provider="groq")
config = llm.generate_from_story(
    "Healthcare app with patients, doctors, and appointments"
)

# 2. Generate data
simulator = DataSimulator(config)
for table_name, df in simulator.generate_all():
    print(f"Generated {len(df)} rows for {table_name}")
    df.to_csv(f"{table_name}.csv", index=False)

SQLAlchemy Seeding (Powerful!)

Directly seed your SQLAlchemy models without writing factories.

from misata import seed_from_sqlalchemy_models
from myapp.models import Base, engine

# Automatically analyzes your models and foreign keys
report = seed_from_sqlalchemy_models(
    engine, 
    Base, 
    default_rows=10_000, 
    create=True, 
    smart_mode=True  # Infers realistic values from column names
)

print(f"Seeded {report.total_rows} rows in {report.duration_seconds}s")

🎯 Business Constraints

Define complex rules that simple random generators can't handle.

from misata import Constraint, Table

timesheets = Table(
    name="timesheets",
    row_count=10000,
    constraints=[
        Constraint(
            name="max_daily_hours",
            type="sum_limit",
            group_by=["employee_id", "date"],
            column="hours",
            value=8.0,
            action="redistribute"  # Automatically fixes violations
        )
    ]
)

πŸ”Œ Providers

Misata supports multiple LLM providers for schema generation.

Provider Env Var Tier Best For
Groq GROQ_API_KEY Free Speed (Recommended)
OpenAI OPENAI_API_KEY Paid Quality
Ollama None Free Privacy (Local)

🏒 Enterprise

Building a platform? Misata Studio is our commercial offering for teams.

  • πŸ–₯️ Visual Schema Editor: Drag-and-drop schema design.
  • πŸ”’ Privacy Filters: PII scanning and masking.
  • πŸ“¦ One-Click Deploy: Docker & Kubernetes ready.
  • 🀝 Support: Dedicated support and custom integration.

Contact Sales for a demo.


Built with ❀️ by Muhammed Rasin

About

High-performance open-source synthetic data engine. Uses LLMs for schema design and vectorized NumPy for deterministic, scalable generation.

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Packages

 
 
 

Languages