How notesjor/cooccurrencefieldsampler works
Project type and purpose
• This is a command-line tool (CLI) written in C# (.NET 10.0) named “CooccurrenceFieldSampler.” Its purpose is to read corpora in various formats, compute co-occurrence statistics on a specified layer, and write results in the chosen export format. There is no GUI or web frontend—this is a standalone console application designed for batch processing of text corpora.
File structure
• Top-level:
• .gitignore, LICENSE, README.md
• CooccurrenceFieldSampler/
– CooccurrenceFieldSampler.csproj
– Program.cs
– publish/ (contains the packaged binaries and a published README)
• There is a single project; no submodules or multiple services.
• Build/deployment artifacts under publish/ indicate a simple “dotnet publish” workflow.
README for additional insights
• Supported import/export formats (>30 formats)
• Various parameters control input folder, output file, filtering thresholds, number of rounds
• Runtime requirement: .NET 10.0 (for non-x64 architectures use “dotnet cfs.dll”)
Main components and interactions
a. Main Components
• CLI Interface / Argument Parser
• Input Format Handlers (one per supported format)
• Co‐occurrence Sampling Engine
• Significance and Frequency Filter
• Output Formatter / Writer
• Logger / Round-by-round tracker
b. Relationships & Data Flow
1. User invokes “cfs.exe” (or “dotnet cfs.dll”) with parameters
2. CLI Interface parses arguments and routes to Input Format Handler
3. Input Handler reads files from the input folder and produces a uniform internal representation (e.g., token streams)
4. Sampling Engine iterates through tokens/contexts, computes co-occurrence counts per round
5. Filter applies minFrequency, minSignificance, minChangeRate thresholds after each round
6. Logger records intermediate results; Writer serializes final co-occurrence data to the chosen export format
c. Architectural Patterns & Principles
• Modular “Strategy” pattern for format handlers (easy to add new formats)
• Single‐responsibility: each module (parsing, sampling, filtering, writing) is separate
• Pipeline pattern: data flows in stages from raw input to final output
d. Key Technologies
• .NET 10 console application
• System.CommandLine (or similar) for argument parsing
• Custom libraries or classes for each format adapter
Essential details to capture
• Iteration loop: Sampling Engine ↔ Filtering Module (multiple rounds)
• Where logging happens (before/after each round)
• Input and output folder paths as contextual notes