Initial commit with Llama API client and docs

Add Python scripts for Llama API chat clients, endpoint testing, and quick tests. Include documentation (README, CONTRIBUTING, 操作指南), license, and .gitignore. Supports multiple endpoints and models for OpenAI-compatible Llama API usage.
2025-09-19 21:44:02 +08:00
parent 4e28c131d2
commit 8a929936ad
18 changed files with 2073 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,201 @@
+# Llama API Client
+
+A Python client for connecting to Llama AI models through OpenAI-compatible API endpoints.
+
+## Features
+
+- 🌐 Support for both internal network and external API endpoints
+- 🤖 Multiple model support (GPT-OSS-120B, DeepSeek-R1-671B, Qwen3-Embedding-8B)
+- 💬 Interactive chat interface with conversation history
+- 🔄 Automatic endpoint testing and failover
+- 🧹 Automatic response cleaning (removes thinking tags and special markers)
+- 📝 Full conversation context management
+
+## Quick Start
+
+### Installation
+
+```bash
+# Clone the repository
+git clone https://github.com/yourusername/llama-api-client.git
+cd llama-api-client
+
+# Install dependencies
+pip install -r requirements.txt
+```
+
+### Basic Usage
+
+```python
+from openai import OpenAI
+
+# Configure API
+API_KEY = "paVrIT+XU1NhwCAOb0X4aYi75QKogK5YNMGvQF1dCyo="
+BASE_URL = "http://192.168.0.6:21180/v1"
+
+# Create client
+client = OpenAI(api_key=API_KEY, base_url=BASE_URL)
+
+# Send request
+response = client.chat.completions.create(
+    model="gpt-oss-120b",
+    messages=[{"role": "user", "content": "Hello!"}],
+    temperature=0.7,
+    max_tokens=200
+)
+
+print(response.choices[0].message.content)
+```
+
+### Run Interactive Chat
+
+```bash
+# Full-featured chat with all endpoints
+python llama_full_api.py
+
+# Internal network only
+python llama_chat.py
+
+# Quick test
+python quick_test.py
+```
+
+## Available Endpoints
+
+### Internal Network (Tested & Working ✅)
+
+| Endpoint | URL | Status |
+|----------|-----|--------|
+| Internal 1 | `http://192.168.0.6:21180/v1` | ✅ Working |
+| Internal 2 | `http://192.168.0.6:21181/v1` | ✅ Working |
+| Internal 3 | `http://192.168.0.6:21182/v1` | ✅ Working |
+| Internal 4 | `http://192.168.0.6:21183/v1` | ❌ Error 500 |
+
+### External Network
+
+| Endpoint | URL | Status |
+|----------|-----|--------|
+| GPT-OSS | `https://llama.theaken.com/v1/gpt-oss-120b` | 🔄 Pending |
+| DeepSeek | `https://llama.theaken.com/v1/deepseek-r1-671b` | 🔄 Pending |
+| General | `https://llama.theaken.com/v1` | 🔄 Pending |
+
+## Project Structure
+
+```
+llama-api-client/
+├── README.md                 # This file
+├── requirements.txt          # Python dependencies
+├── 操作指南.md               # Chinese operation guide
+├── llama_full_api.py        # Full-featured chat client
+├── llama_chat.py            # Internal network chat client
+├── local_api_test.py        # Endpoint testing tool
+├── quick_test.py            # Quick connection test
+├── test_all_models.py       # Model testing script
+└── demo_chat.py             # Demo chat with fallback
+```
+
+## Chat Commands
+
+During chat sessions, you can use these commands:
+
+- `exit` or `quit` - End the conversation
+- `clear` - Clear conversation history
+- `model` - Switch between available models
+
+## Configuration
+
+### API Key
+```python
+API_KEY = "paVrIT+XU1NhwCAOb0X4aYi75QKogK5YNMGvQF1dCyo="
+```
+
+### Available Models
+- `gpt-oss-120b` - GPT Open Source 120B parameters
+- `deepseek-r1-671b` - DeepSeek R1 671B parameters
+- `qwen3-embedding-8b` - Qwen3 Embedding 8B parameters
+
+## Troubleshooting
+
+### Issue: 502 Bad Gateway
+**Cause**: External API server is offline  
+**Solution**: Use internal network endpoints
+
+### Issue: Connection Error
+**Cause**: Not on internal network or incorrect IP  
+**Solution**: 
+1. Verify network connectivity: `ping 192.168.0.6`
+2. Check firewall settings
+3. Ensure you're on the same network
+
+### Issue: Encoding Error
+**Cause**: Windows terminal encoding issues  
+**Solution**: Use English for conversations or modify terminal encoding
+
+### Issue: Response Contains Special Markers
+**Description**: Responses may contain `<think>`, `<|channel|>` tags  
+**Solution**: The client automatically removes these markers
+
+## Response Cleaning
+
+The client automatically removes these special markers from AI responses:
+- `<think>...</think>` - Thinking process
+- `<|channel|>...<|message|>` - Channel markers
+- `<|end|>`, `<|start|>` - End/start markers
+
+## Requirements
+
+- Python 3.7+
+- openai>=1.0.0
+- requests (optional, for direct API calls)
+
+## Development
+
+### Testing Connection
+```python
+python -c "from openai import OpenAI; client = OpenAI(api_key='YOUR_KEY', base_url='YOUR_URL'); print(client.chat.completions.create(model='gpt-oss-120b', messages=[{'role': 'user', 'content': 'test'}], max_tokens=5).choices[0].message.content)"
+```
+
+### Adding New Endpoints
+Edit `ENDPOINTS` dictionary in `llama_full_api.py`:
+```python
+ENDPOINTS = {
+    "internal": [
+        {
+            "name": "New Endpoint",
+            "url": "http://new-endpoint/v1",
+            "models": ["gpt-oss-120b"]
+        }
+    ]
+}
+```
+
+## License
+
+MIT License - See LICENSE file for details
+
+## Contributing
+
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
+
+## Support
+
+For issues or questions:
+1. Check the [操作指南.md](操作指南.md) for detailed Chinese documentation
+2. Open an issue on GitHub
+3. Contact the API administrator for server-related issues
+
+## Acknowledgments
+
+- Built with OpenAI Python SDK
+- Compatible with OpenAI API format
+- Supports multiple Llama model variants
+
+---
+
+**Last Updated**: 2025-09-19  
+**Version**: 1.0.0  
+**Status**: Internal endpoints working, external endpoints pending