merge: Sync latest main changes into development (Tasks 1.3, 2.1, 2.2)

Author: yusufkaraaslan

CLAUDE.md

@@ -15,15 +15,15 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 - **✅ CI/CD Fixed**: All 5 test matrix jobs passing (Ubuntu + macOS, Python 3.10-3.12)
 - **📚 Documentation Complete**: README, CHANGELOG, FUTURE_RELEASES.md all updated
 - **🚀 Unified CLI**: Single `skill-seekers` command with Git-style subcommands
-- **🧪 Test Coverage**: 379 tests passing, 39% coverage
+- **🧪 Test Coverage**: 391 tests passing, 39% coverage
 - **🌐 Community**: GitHub Discussion, Release notes, announcements published
 
 **🚀 Unified Multi-Source Scraping (v2.0.0)**
 - **NEW**: Combine documentation + GitHub + PDF in one skill
 - **NEW**: Automatic conflict detection between docs and code
 - **NEW**: Rule-based and AI-powered merging
 - **NEW**: 5 example unified configs (React, Django, FastAPI, Godot, FastAPI-test)
-- **Status**: ⚠️ 12 unified tests need fixes (core functionality stable)
+- **Status**: ✅ All 22 unified tests passing (18 core + 4 MCP integration)
 
 **✅ Community Response (H1 Group):**
 - **Issue #8 Fixed** - Added BULLETPROOF_QUICKSTART.md and TROUBLESHOOTING.md for beginners
@@ -40,16 +40,17 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 - 📝 Multi-source configs: django_unified, fastapi_unified, fastapi_unified_test, godot_unified, react_unified
 - 📝 Test/Example configs: godot_github, react_github, python-tutorial-test, example_pdf, test-manual
 
-**📋 Next Up (Post-PyPI v2.0.0):**
-- **✅ DONE**: PyPI publication complete
+**📋 Next Up (Post-v2.1.0):**
+- **✅ DONE**: PyPI publication complete (v2.0.0)
 - **✅ DONE**: CI/CD fixed - all checks passing
 - **✅ DONE**: Documentation updated (README, CHANGELOG, FUTURE_RELEASES.md)
-- **Priority 1**: Fix 12 failing unified tests in tests/test_unified.py
-  - ConfigValidator expecting dict instead of file path
-  - ConflictDetector expecting dict pages, not list
+- **✅ DONE**: Quality Assurance + Race Condition Fixes (v2.1.0)
+- **✅ DONE**: All critical bugs fixed (Issues #190, #192, #193)
+- **✅ DONE**: Test suite stabilized (391 tests passing)
+- **✅ DONE**: Unified tests fixed (all 22 passing)
+- **Priority 1**: Review and merge open PRs (#195, #196, #197, #198)
 - **Priority 2**: Task H1.3 - Create example project folder
 - **Priority 3**: Task A3.1 - GitHub Pages site (skillseekersweb.com)
-- **Priority 4**: Task J1.1 - Install MCP package for testing
 
 **📊 Roadmap Progress:**
 - 134 tasks organized into 22 feature groups
@@ -325,12 +326,13 @@ Skill_Seekers/
 │       │   └── conflict_detector.py # Conflict detection
 │       └── mcp/                # MCP server integration
 │           └── server.py
-├── tests/                      # Test suite (379 tests passing)
+├── tests/                      # Test suite (391 tests passing)
 │   ├── test_scraper_features.py
 │   ├── test_config_validation.py
 │   ├── test_integration.py
 │   ├── test_mcp_server.py
-│   ├── test_unified.py         # (12 tests need fixes)
+│   ├── test_unified.py         # Unified scraping tests (18 tests)
+│   ├── test_unified_mcp_integration.py  # (4 tests)
 │   └── ...
 ├── configs/                    # Preset configurations (24 configs)
 │   ├── godot.json
@@ -743,11 +745,11 @@ The correct command uses the local `cli/package_skill.py` in the repository root
 - ✅ `claude-code.json` - Claude Code documentation **NEW!**
 
 ### Unified Multi-Source Configs (5 configs - **NEW v2.0!**)
-- ⚠️ `react_unified.json` - React (docs + GitHub + code analysis)
-- ⚠️ `django_unified.json` - Django (docs + GitHub + code analysis)
-- ⚠️ `fastapi_unified.json` - FastAPI (docs + GitHub + code analysis)
-- ⚠️ `fastapi_unified_test.json` - FastAPI test config
-- ⚠️ `godot_unified.json` - Godot (docs + GitHub + code analysis)
+- ✅ `react_unified.json` - React (docs + GitHub + code analysis)
+- ✅ `django_unified.json` - Django (docs + GitHub + code analysis)
+- ✅ `fastapi_unified.json` - FastAPI (docs + GitHub + code analysis)
+- ✅ `fastapi_unified_test.json` - FastAPI test config
+- ✅ `godot_unified.json` - Godot (docs + GitHub + code analysis)
 
 ### Test/Example Configs (5 configs)
 - 📝 `godot_github.json` - GitHub-only scraping example
@@ -756,8 +758,8 @@ The correct command uses the local `cli/package_skill.py` in the repository root
 - 📝 `example_pdf.json` - PDF extraction example
 - 📝 `test-manual.json` - Manual testing config
 
-**Note:** ⚠️ = Unified configs have 12 failing tests that need fixing
-**Last verified:** November 11, 2025 (v2.0.0 PyPI release)
+**Note:** All configs verified and working! Unified configs fully tested with 22 passing tests.
+**Last verified:** November 29, 2025 (Post-v2.1.0 bug fixes)
 
 ## Additional Documentation
 
@@ -789,7 +791,7 @@ The correct command uses the local `cli/package_skill.py` in the repository root
 - ✅ **Modern Python Packaging**: pyproject.toml, src/ layout, entry points
 - ✅ **Unified CLI**: Single `skill-seekers` command with Git-style subcommands
 - ✅ **CI/CD Working**: All 5 test matrix jobs passing (Ubuntu + macOS, Python 3.10-3.12)
-- ✅ **Test Coverage**: 379 tests passing, 39% coverage
+- ✅ **Test Coverage**: 391 tests passing, 39% coverage
 - ✅ **Documentation**: Complete user and technical documentation
 
 **Architecture:**
@@ -801,7 +803,7 @@ The correct command uses the local `cli/package_skill.py` in the repository root
 
 **Development Workflow:**
 1. **Install**: `pip install -e .` (editable mode for development)
-2. **Run tests**: `pytest tests/` (379 tests)
+2. **Run tests**: `pytest tests/` (391 tests)
 3. **Build package**: `uv build` or `python -m build`
 4. **Publish**: `uv publish` (PyPI)
 

README.md

@@ -86,7 +86,7 @@ Skill Seeker is an automated tool that transforms documentation websites, GitHub
 - ✅ **Caching System** - Scrape once, rebuild instantly
 
 ### ✅ Quality Assurance
-- ✅ **Fully Tested** - 379 tests with comprehensive coverage
+- ✅ **Fully Tested** - 391 tests with comprehensive coverage
 
 ---
 

docs/CLAUDE.md

@@ -326,6 +326,40 @@ print(soup.select_one('main'))
 print(soup.select_one('div[role="main"]'))
 ```
 
+## Running Tests
+
+**IMPORTANT: You must install the package before running tests**
+
+```bash
+# 1. Install package in editable mode (one-time setup)
+pip install -e .
+
+# 2. Run all tests
+pytest
+
+# 3. Run specific test files
+pytest tests/test_config_validation.py
+pytest tests/test_github_scraper.py
+
+# 4. Run with verbose output
+pytest -v
+
+# 5. Run with coverage report
+pytest --cov=src/skill_seekers --cov-report=html
+```
+
+**Why install first?**
+- Tests import from `skill_seekers.cli` which requires the package to be installed
+- Modern Python packaging best practice (PEP 517/518)
+- CI/CD automatically installs with `pip install -e .`
+- conftest.py will show helpful error if package not installed
+
+**Test Coverage:**
+- 391+ tests passing
+- 39% code coverage
+- All core features tested
+- CI/CD tests on Ubuntu + macOS with Python 3.10-3.12
+
 ## Troubleshooting
 
 **No content extracted**: Check `main_content` selector. Common values: `article`, `main`, `div[role="main"]`, `div.content`

src/skill_seekers/cli/github_scraper.py

@@ -31,6 +31,13 @@
     print("Error: PyGithub not installed. Run: pip install PyGithub")
     sys.exit(1)
 
+# Configure logging FIRST (before using logger)
+logging.basicConfig(
+    level=logging.INFO,
+    format='%(asctime)s - %(levelname)s - %(message)s'
+)
+logger = logging.getLogger(__name__)
+
 # Import code analyzer for deep code analysis
 try:
     from .code_analyzer import CodeAnalyzer
@@ -39,13 +46,6 @@
     CODE_ANALYZER_AVAILABLE = False
     logger.warning("Code analyzer not available - deep analysis disabled")
 
-# Configure logging
-logging.basicConfig(
-    level=logging.INFO,
-    format='%(asctime)s - %(levelname)s - %(message)s'
-)
-logger = logging.getLogger(__name__)
-
 
 class GitHubScraper:
     """

tests/conftest.py

@@ -2,11 +2,28 @@
 Pytest configuration for tests.
 
 Configures anyio to only use asyncio backend (not trio).
+Checks that the skill_seekers package is installed before running tests.
 """
 
+import sys
 import pytest
 
 
+def pytest_configure(config):
+    """Check if package is installed before running tests."""
+    try:
+        import skill_seekers
+    except ModuleNotFoundError:
+        print("\n" + "=" * 70)
+        print("ERROR: skill_seekers package not installed")
+        print("=" * 70)
+        print("\nPlease install the package in editable mode first:")
+        print("  pip install -e .")
+        print("\nOr activate your virtual environment if you already installed it.")
+        print("=" * 70 + "\n")
+        sys.exit(1)
+
+
 @pytest.fixture(scope="session")
 def anyio_backend():
     """Override anyio backend to only use asyncio (not trio)."""

tests/test_constants.py

@@ -109,14 +109,14 @@ class TestConstantsUsage(unittest.TestCase):
 
     def test_doc_scraper_imports_constants(self):
         """Test that doc_scraper imports and uses constants."""
-        from cli import doc_scraper
+        from skill_seekers.cli import doc_scraper
         # Check that doc_scraper can access the constants
         self.assertTrue(hasattr(doc_scraper, 'DEFAULT_RATE_LIMIT'))
         self.assertTrue(hasattr(doc_scraper, 'DEFAULT_MAX_PAGES'))
 
     def test_estimate_pages_imports_constants(self):
         """Test that estimate_pages imports and uses constants."""
-        from cli import estimate_pages
+        from skill_seekers.cli import estimate_pages
         # Verify function signature uses constants
         import inspect
         sig = inspect.signature(estimate_pages.estimate_pages)
@@ -125,7 +125,7 @@ def test_estimate_pages_imports_constants(self):
     def test_enhance_skill_imports_constants(self):
         """Test that enhance_skill imports constants."""
         try:
-            from cli import enhance_skill
+            from skill_seekers.cli import enhance_skill
             # Check module loads without errors
             self.assertIsNotNone(enhance_skill)
         except (ImportError, SystemExit) as e:
@@ -135,7 +135,7 @@ def test_enhance_skill_imports_constants(self):
 
     def test_enhance_skill_local_imports_constants(self):
         """Test that enhance_skill_local imports constants."""
-        from cli import enhance_skill_local
+        from skill_seekers.cli import enhance_skill_local
         self.assertIsNotNone(enhance_skill_local)
 
 
@@ -144,7 +144,7 @@ class TestConstantsExports(unittest.TestCase):
 
     def test_all_exports_exist(self):
         """Test that all items in __all__ exist."""
-        from cli import constants
+        from skill_seekers.cli import constants
         self.assertTrue(hasattr(constants, '__all__'))
         for name in constants.__all__:
             self.assertTrue(
@@ -154,7 +154,7 @@ def test_all_exports_exist(self):
 
     def test_all_exports_count(self):
         """Test that __all__ has expected number of exports."""
-        from cli import constants
+        from skill_seekers.cli import constants
         # We defined 18 constants (added DEFAULT_ASYNC_MODE)
         self.assertEqual(len(constants.__all__), 18)
 

tests/test_llms_txt_detector.py

@@ -6,7 +6,7 @@ def test_detect_llms_txt_variants():
     """Test detection of llms.txt file variants"""
     detector = LlmsTxtDetector("https://hono.dev/docs")
 
-    with patch('cli.llms_txt_detector.requests.head') as mock_head:
+    with patch('skill_seekers.cli.llms_txt_detector.requests.head') as mock_head:
         mock_response = Mock()
         mock_response.status_code = 200
         mock_head.return_value = mock_response
@@ -22,7 +22,7 @@ def test_detect_no_llms_txt():
     """Test detection when no llms.txt file exists"""
     detector = LlmsTxtDetector("https://example.com/docs")
 
-    with patch('cli.llms_txt_detector.requests.head') as mock_head:
+    with patch('skill_seekers.cli.llms_txt_detector.requests.head') as mock_head:
         mock_response = Mock()
         mock_response.status_code = 404
         mock_head.return_value = mock_response
@@ -36,7 +36,7 @@ def test_url_parsing_with_complex_paths():
     """Test URL parsing handles non-standard paths correctly"""
     detector = LlmsTxtDetector("https://example.com/docs/v2/guide")
 
-    with patch('cli.llms_txt_detector.requests.head') as mock_head:
+    with patch('skill_seekers.cli.llms_txt_detector.requests.head') as mock_head:
         mock_response = Mock()
         mock_response.status_code = 200
         mock_head.return_value = mock_response
@@ -55,7 +55,7 @@ def test_detect_all_variants():
     """Test detecting all llms.txt variants"""
     detector = LlmsTxtDetector("https://hono.dev/docs")
 
-    with patch('cli.llms_txt_detector.requests.head') as mock_head:
+    with patch('skill_seekers.cli.llms_txt_detector.requests.head') as mock_head:
         # Mock responses for different variants
         def mock_response(url, **kwargs):
             response = Mock()