fix: Semantic Scholar 429 (#25), auto-detect .bib (#9), fix docs (#6 #8 #11 #12 #15 #18)

- #25: return [] silently on 429 instead of logging WARNING
- #9: auto-detect input_type='bib' when file has .bib extension
- #8: remove false claim that keywords route to specific data sources
- #18: clarify entries must be blank-line separated in basic_usage docs
- #11: update api/core.rst with use_google_scholar param
- #12: remove nonexistent requirements-dev.txt from CONTRIBUTING.md
- #15: remove false claim about black formatting from CONTRIBUTING.md
- #6: update data sources description to reflect actual behaviour
- docs: document --google-scholar, direct string input, and stdin

Closes #25
Closes #9
Closes #8
Closes #11
Closes #12
Closes #15
Closes #18

Author: Ang

CONTRIBUTING.md

@@ -42,10 +42,6 @@ This project adheres to the Contributor Covenant [Code of Conduct](CODE_OF_CONDU
 ```bash
 # Install the package in editable mode with development dependencies
 pip install -e ".[dev]"
-
-# Or install from requirements files
-pip install -r requirements.txt
-pip install -r requirements-dev.txt
 ```
 
 ### Verify Installation
@@ -151,14 +147,6 @@ We follow [PEP 8](https://pep8.org/) style guide. Key points:
 - Use descriptive variable and function names
 - Add docstrings to all public functions and classes
 
-### Code Formatting
-
-We use `black` for code formatting:
-
-```bash
-black onecite tests
-```
-
 ### Type Hints
 
 Use type hints where possible:

docs/advanced_usage.rst

@@ -51,29 +51,9 @@ Process multiple files sequentially::
 Working with Different Data Sources
 ------------------------------------
 
-OneCite routes queries to data sources based on content type:
+OneCite queries multiple data sources (CrossRef, PubMed, arXiv, Semantic Scholar, Google Books, and others) and selects the best match. All sources are tried for every reference — you do not need to configure routing manually::
 
-**For Biomedical Literature**
-
-Add search terms related to medicine, biology, or health::
-
-    onecite process medical_refs.txt
-
-This will prioritize PubMed when available.
-
-**For Computer Science**
-
-Add search terms related to CS topics::
-
-    onecite process cs_refs.txt
-
-This will prioritize DBLP and arXiv.
-
-**For General Academic Work**
-
-Mixed references will use CrossRef and Semantic Scholar::
-
-    onecite process general_refs.txt
+    onecite process references.txt
 
 Custom Templates
 ----------------

docs/api/core.rst

@@ -18,16 +18,18 @@ The primary function for processing citations.
         input_type: str,
         template_name: str,
         output_format: str,
-        interactive_callback: Callable[[List[Dict]], int]
+        interactive_callback: Callable[[List[Dict]], int],
+        use_google_scholar: bool = False,
     ) -> Dict[str, Any]
 
 **Parameters:**
 
 - ``input_content`` (str): The reference content to process
-- ``input_type`` (str): Type of input - "txt" or "bib" (required)
-- ``template_name`` (str): Template name to use (e.g., "journal_article_full") (required)
-- ``output_format`` (str): Output format - "bibtex", "apa", or "mla" (required)
+- ``input_type`` (str): Type of input - ``"txt"`` or ``"bib"`` (required)
+- ``template_name`` (str): Template name to use (e.g., ``"journal_article_full"``) (required)
+- ``output_format`` (str): Output format - ``"bibtex"``, ``"apa"``, or ``"mla"`` (required)
 - ``interactive_callback`` (Callable): Function to handle ambiguous matches. Takes a list of candidate dicts and returns the selected index (0-based), or -1 to skip (required)
+- ``use_google_scholar`` (bool): Enable Google Scholar as an additional data source. Requires the optional ``scholarly`` package. Default is ``False``.
 
 **Returns:**
 

docs/basic_usage.rst

@@ -13,12 +13,19 @@ Supported Input Formats
 
 **Plain Text (.txt)**
 
-A simple text file with one reference per line or separated by blank lines::
+A text file where each reference is separated by a **blank line**::
 
     10.1038/nature14539
+
     Vaswani et al., 2017, Attention is all you need
+
     Smith (2020) Neural Architecture Search
 
+.. note::
+
+   Each entry must be separated by at least one blank line. Adjacent lines
+   within the same block are treated as a single entry.
+
 **BibTeX (.bib)**
 
 Standard BibTeX format files::
@@ -108,6 +115,25 @@ Suppress verbose output::
 
     onecite process input.txt --quiet
 
+**Google Scholar (--google-scholar)**
+
+Enable Google Scholar as an additional data source (requires the optional ``scholarly`` package)::
+
+    onecite process input.txt --google-scholar
+
+**Direct String Input**
+
+Pass a reference string directly instead of a file::
+
+    onecite process "10.1038/nature14539"
+    onecite process "Attention is all you need, Vaswani et al., NIPS 2017"
+
+**Stdin Input**
+
+Read from standard input using ``-``::
+
+    echo "10.1038/nature14539" | onecite process -
+
 **Help (--help)**
 
 Display help information::
@@ -126,6 +152,7 @@ Example 1: Process a BibTeX File
     onecite process my_references.bib -o clean_references.bib --quiet
 
 This will read ``my_references.bib``, enhance the entries, and save to ``clean_references.bib``.
+The ``--input-type`` flag is optional for ``.bib`` files — OneCite detects the format automatically.
 
 Example 2: Convert to APA Format
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

onecite/cli.py

@@ -131,6 +131,8 @@ def process_command(args: "argparse.Namespace") -> int:
         elif os.path.exists(args.input_file):
             with open(args.input_file, 'r', encoding='utf-8') as f:
                 input_content = f.read()
+            if args.input_type == 'txt' and args.input_file.lower().endswith('.bib'):
+                args.input_type = 'bib'
         else:
             input_content = args.input_file
 

onecite/pipeline.py

@@ -931,6 +931,10 @@ def _search_semantic_scholar(self, query: str, limit: int = 5) -> List[Dict]:
 
             response = requests.get(url, params=params, timeout=10)
 
+            if response.status_code == 429:
+                self.logger.debug("Semantic Scholar rate-limited (429); skipping for this query.")
+                return []
+            
             if response.status_code == 200:
                 data = response.json()
                 papers = data.get('data', [])

tests/test_cli.py

@@ -94,6 +94,21 @@ def _ns(**overrides):
 
     # -- Missing / bad input --------------------------------------------------
 
+    def test_bib_file_auto_detected(self, tmp_path, capsys):
+        """fix #9: .bib extension should auto-set input_type to 'bib'."""
+        inf = tmp_path / "refs.bib"
+        inf.write_text("@article{A, title={T}}", encoding="utf-8")
+        captured = {}
+
+        def _fake(*, input_type, **kw):
+            captured['input_type'] = input_type
+            return {"results": ["OK"], "report": {"total": 1, "succeeded": 1, "failed_entries": []}}
+
+        with patch("onecite.cli.process_references", side_effect=_fake):
+            cli.process_command(self._ns(input_file=str(inf), quiet=True))
+
+        assert captured['input_type'] == 'bib'
+
     def test_google_scholar_flag_passed_through(self, capsys):
         """fix #10: --google-scholar flag must be forwarded to process_references."""
         captured = {}

tests/test_pipeline_unit.py

@@ -691,6 +691,14 @@ def test_strip_html(self):
         assert e._strip_html_tags("Human-level <i>control</i> &amp; learning") == \
                "Human-level control & learning"
 
+    def test_semantic_scholar_429_returns_empty(self):
+        """fix #25: 429 from Semantic Scholar must return [] without raising."""
+        ident = IdentifierModule()
+        resp = DummyResponse(status_code=429, json_data={})
+        with patch("onecite.pipeline.requests.get", return_value=resp):
+            result = ident._search_semantic_scholar("attention is all you need")
+        assert result == []
+
     def test_crossref_request_has_user_agent_and_mailto(self):
         """fix #21: _get_crossref_metadata must send User-Agent and mailto."""
         e = EnricherModule(use_google_scholar=False)