How deep-think-project/server works
Project type and purpose
• This is a full-stack (API-only) web service built with Django.
• Purpose: ingest raw text or URL, classify each sentence as “clear” or “ambiguous” via OpenAI GPT, enrich ambiguous sentences using Perplexity Sonar, then return a detailed JSON response.
File structure
• Top-level:
• deepthink_server/ (Django project folder)
– deepthink_server/ (settings, URLs, ASGI/WGSI entry points)
– main_app/ (core business logic: views, utils, models, URL routing, system prompts)
– output/ (six subfolders for intermediate debug files)
– manage.py
• requirements.txt, example.json, README.md
• Key patterns: layered MVC (Django), pipeline stages (raw → indexed → GPT → Sonar → merged)
README for insights
• Describes API endpoint, input/output JSON schema
• Lists tech stack: Django, requests + BeautifulSoup (web scraping), spaCy (nlp), openai, dotenv
• Notes debug files under output/1-raw … output/6-merge_gpt_sonar
Main components
a. External clients
• Web or script POST → Django API
b. Django API Layer
• ASGI/WGSI server (uvicorn/gunicorn)
• URL router → main_app.views.analyze()
c. Pre-processing
• Input handler (URL fetch via requests + BeautifulSoup)
• spaCy pipeline (ko_core_news_sm) for sentence segmentation
d. AI services
• OpenAI GPT API → Classify sentences
• Perplexity Sonar API → Fetch supporting references
e. Merge & Response
• utils.merge_gpt_and_sonar()
• JSON serializer → HTTP response
f. Debug storage
• output/1-raw, 2-indexed, 3-first_gpt_output, 4-for_sonar, 5-sonar_output, 6-merge_gpt_sonar
Relationships & data flow
1. Client → HTTP POST → Django ASGI
2. Django view calls Input Handler
3. If URL: requests → BeautifulSoup → raw text → spaCy
4. spaCy → list of sentences → for-loop
5. Sentence → OpenAI GPT API → returns classification & reason
6. Ambiguous sentences → Perplexity Sonar API → returns references
7. utils merges both outputs → final JSON
8. Debug files written at each stage
9. JSON → HTTP response → Client
Architectural patterns & principles
• Layered (presentation → application → integration)
• Pipeline stages saved for reproducibility (debug)
• Single-responsibility in utils, views, system_prompt files
• Environment-based config (.env via dotenv)
• Clear separation of external (AI services) vs. internal components