A comprehensive, state-of-the-art framework for detecting and analyzing bias in Small Language Models (SLMs) and Large Language Models (LLMs). Our 3-phase methodology provides corpus generation, model auditing, and result analysis for researchers, developers, and organizations committed to responsible AI development.
Everything you need to detect, measure, and analyze bias in AI language models
Rich terminal UI with guided workflows, smart auto-discovery, and comprehensive help system. Modern Typer-based CLI with cross-platform launchers.
Actual API timing with 1.4x safety buffer for accurate planning. Multi-prompt averaging for reliable time estimates with dynamic ETA updates.
NVIDIA CUDA support for 5-10x faster model inference. Automatic GPU detection and configuration with graceful fallback to CPU-only mode.
Graceful handling of interruptions with automatic session recovery. Persistent session state across restarts with resume from exact point of interruption.
Rich progress bars with individual test timing metrics. Real-time performance monitoring with colorful, informative status indicators.
Detailed performance metrics and bias analysis reports. Statistical analysis with visualization and export capabilities for further research.
Create your own bias detection datasets using our CorpusGen tool. Customize names, professions, traits, and templates to audit specific biases in your domain.
Apache 2.0 licensed with deterministic corpus generation. Publish datasets on Zenodo for peer review and ensure reproducible research results.
A systematic approach to bias detection with modular, scalable design
Configure custom bias categories with JSON-driven specifications. Generate balanced, reproducible datasets using our systematic names × professions × traits × templates approach.
Execute bias tests against language models, collect responses, and measure bias indicators across multiple categories.
Analyze results, generate comprehensive reports, and visualize bias patterns with statistical significance testing.
Follow these simple steps to start detecting bias in your AI models
Get the latest version of EquiLens from GitHub
git clone
https://github.com/Life-Experimentalist/EquiLens.git
cd EquiLens
Set up the Python environment with UV package manager
# Install UV curl -LsSf
https://astral.sh/uv/install.sh | sh # Install
dependencies uv sync
Install and configure Ollama for model inference
# Install Ollama curl -fsSL
https://ollama.ai/install.sh | sh # Pull a model
ollama pull llama2
Execute the interactive CLI to start bias detection
# Windows .\scripts\equilens.bat #
Unix/Linux/macOS ./scripts/equilens.sh
Interactive demonstration of the bias detection process
Create balanced, reproducible bias detection datasets with our open-source corpus generation tool
Systematically combines names × professions × traits × templates
Equal representation across gender, traits, and professions
Extendable to new professions, names, and bias categories
Deterministic corpus generation for peer review
| Column | Description |
|---|---|
| comparison_type | Audit category (e.g., gender_bias) |
| name | The chosen first name |
| name_category | Name group (e.g., Male/Female) |
| profession | Profession label (e.g., Engineer, Nurse) |
| trait | Trait word (e.g., Logical, Caring) |
| trait_category | Competence or Social classification |
| template_id | ID of the sentence template used |
| full_prompt_text | Final generated sentence |
Execute comprehensive bias tests against Small and Large Language Models with real-time monitoring
Accurate time estimation with 1.4x safety buffer and multi-prompt averaging for reliable planning
NVIDIA CUDA support for 5-10x faster inference with automatic detection and graceful CPU fallback
Graceful handling of interruptions with persistent session state and resume from exact point
Rich progress bars with individual test timing, real-time performance metrics, and status indicators
Production-ready stable auditor plus enhanced research auditor with advanced features
Containerized Ollama with GPU passthrough, automatic service detection, and persistent storage
Model Auditor will be available in the next release with comprehensive LLM/SLM testing capabilities
Get NotifiedComprehensive statistical analysis and visualization of bias detection results
Advanced statistical methods for bias quantification, significance testing, and correlation analysis
Dynamic charts, heatmaps, and dashboards for intuitive bias pattern exploration
Automated generation of publication-ready reports with citations and methodology details
Multiple format support including CSV, JSON, PDF, and LaTeX for academic publishing
Analysis framework will complete the EquiLens pipeline, providing comprehensive bias insights
View Roadmap