A comprehensive, state-of-the-art framework for detecting and analyzing bias in Small Language Models (SLMs) and Large Language Models (LLMs). Our 3-phase methodology provides corpus generation, model auditing, and result analysis for researchers, developers, and organizations committed to responsible AI development.
Everything you need to detect, measure, and analyze bias in AI language models
Rich terminal UI with guided workflows, smart auto-discovery, and comprehensive help system. Modern Typer-based CLI with cross-platform launchers.
Actual API timing with 1.4x safety buffer for accurate planning. Multi-prompt averaging for reliable time estimates with dynamic ETA updates.
NVIDIA CUDA support for 5-10x faster model inference. Automatic GPU detection and configuration with graceful fallback to CPU-only mode.
Graceful handling of interruptions with automatic session recovery. Persistent session state across restarts with resume from exact point of interruption.
Rich progress bars with individual test timing metrics. Real-time performance monitoring with colorful, informative status indicators.
Detailed performance metrics and bias analysis reports. Statistical analysis with visualization and export capabilities for further research.
Create your own bias detection datasets using our CorpusGen tool. Customize names, professions, traits, and templates to audit specific biases in your domain.
Apache 2.0 licensed with deterministic corpus generation. Publish datasets on Zenodo for peer review and ensure reproducible research results.
FastAPI + Jinja2 + Alpine.js dashboard served at http://localhost:8000 — no Gradio, no separate process. SSE real-time job logs, inline result viewing, periodic automated backups, and a full OpenAPI 3.1 contract.
A systematic approach to bias detection with modular, scalable design
Configure custom bias categories with JSON-driven specifications. Generate balanced, reproducible datasets using our systematic names × professions × traits × templates approach.
Execute bias tests against language models, collect responses, and measure bias indicators across multiple categories.
Analyze results, generate comprehensive reports, and visualize bias patterns with statistical significance testing.
Follow these simple steps to start detecting bias in your AI models
Get the latest version of EquiLens from GitHub
git clone
https://github.com/Life-Experimentalist/EquiLens.git
cd EquiLens
Set up the Python environment with UV package manager
# Install UV curl -LsSf
https://astral.sh/uv/install.sh | sh # Install
dependencies uv sync
Install and configure Ollama for model inference
# Install Ollama curl -fsSL
https://ollama.ai/install.sh | sh # Pull a model
ollama pull llama3.2
Open the built-in web dashboard or run a bias audit from the CLI
# Launch dashboard + API server at
http://localhost:8000 uv run equilens web # Or
run a CLI audit directly uv run equilens audit
--model llama3.2
Interactive demonstration of the bias detection process
Create balanced, reproducible bias detection datasets with our open-source corpus generation tool
Systematically combines names × professions × traits × templates
Equal representation across gender, traits, and professions
Extendable to new professions, names, and bias categories
Deterministic corpus generation for peer review
| Column | Description |
|---|---|
| comparison_type | Audit category (e.g., gender_bias) |
| name | The chosen first name |
| name_category | Name group (e.g., Male/Female) |
| profession | Profession label (e.g., Engineer, Nurse) |
| trait | Trait word (e.g., Logical, Caring) |
| trait_category | Competence or Social classification |
| template_id | ID of the sentence template used |
| full_prompt_text | Final generated sentence |
Execute comprehensive bias tests against Small and Large Language Models with real-time monitoring
Accurate time estimation with 1.4x safety buffer and multi-prompt averaging for reliable planning
NVIDIA CUDA support for 5-10x faster inference with automatic detection and graceful CPU fallback
Graceful handling of interruptions with persistent session state and resume from exact point
Rich progress bars with individual test timing, real-time performance metrics, and status indicators
Production-ready stable auditor plus enhanced research auditor with advanced features
Containerized Ollama with GPU passthrough, automatic service detection, and persistent storage
v2.2.0 ships a full HTML dashboard (FastAPI + Jinja2 +
Alpine.js) with SSE real-time job logs, periodic
automated backups, and an OpenAPI 3.1 contract —
accessible at http://localhost:8000 via
uv run equilens web
Comprehensive statistical analysis and visualization of bias detection results
Advanced statistical methods for bias quantification, significance testing, and correlation analysis
Dynamic charts, heatmaps, and dashboards for intuitive bias pattern exploration
Automated generation of publication-ready reports with citations and methodology details
Multiple format support including CSV, JSON, PDF, and LaTeX for academic publishing
All three phases are now released. The analysis engine
generates HTML and Markdown reports, performs
statistical significance testing, and produces
AI-assisted insights via the
equilens analyze
CLI command.
B.Tech final-year research project at Amrita Vishwa Vidyapeetham — published on Zenodo with a citable DOI
As SLMs gain adoption in hiring, healthcare, and education, bias in their outputs poses real ethical and legal risk — but most auditing tools require white-box access or cloud APIs.
Design a framework that audits any Ollama-compatible model purely through prompts, produces statistically validated bias scores, and is deployable by a solo researcher without GPU cluster access.
Built a three-phase pipeline (corpus generation → model auditing → statistical analysis) with 94,200+ curated prompt variants spanning gender, race, occupation, and sentiment, backed by a FastAPI job system and dual CLI/web interface.
Detected statistically significant gender bias (Cohen's d > 0.4) and occupational stereotyping in multiple SLMs. Published on Zenodo with a citable DOI and one-command Docker deployment.
What's shipped and what's coming next
Corpus generation, model auditing, and statistical analysis — all three phases complete and published in v1.0.0.
FastAPI + Jinja2 + Alpine.js dashboard at
localhost:8000
— no Gradio, no separate server. SSE real-time job
logs and OpenAPI 3.1.
One-command containerized deployment with GPU passthrough, persistent storage, and automatic Ollama service detection.
Browser UI for building and editing bias detection
corpora without touching
word_lists.json
directly.
Side-by-side bias score comparison across multiple Ollama models in a single dashboard view, with differential bias charts.
Extend the auditor to run directly against HuggingFace Transformers models, without requiring an Ollama API wrapper.
Additional word lists for cultural and linguistic contexts beyond English — Hindi, Tamil, Spanish, and other high-usage languages.
Want to contribute? Read the contributing guide or open an issue.