gitwtfhub

wtf is sampler?

dennybritz/sampler — explained in plain English

Analysis updated 2026-08-04 · repo last pushed 2014-05-05

1ScalaAudience · researcherComplexity · 4/5DormantSetup · moderate

TL;DR

A Scala tool that uses statistical sampling to estimate how likely certain facts are true, given a set of rules and evidence. Designed for DeepDive pipelines that extract knowledge from text.

Mindmap

mindmap
  root((repo))
    What it does
      Probabilistic reasoning
      Estimates fact likelihood
      Learns rule weights
    Inputs
      Variables file
      Factors file
      Weights file
    Outputs
      Sampled values
      Learned weights
    Use cases
      Knowledge base building
      Text relationship extraction
      Corporate acquisition tracking
    Tech stack
      Scala
      JVM
      Tab-separated files
    Audience
      Researchers
      Information extraction engineers

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

Why would anyone build with this?

REASON 1

Populate a database of corporate acquisitions from news articles using inference rules.

REASON 2

Compute confidence scores for relationships extracted from text in a DeepDive pipeline.

REASON 3

Tune rule weights by running gradient descent over a web of interconnected evidence.

What's in the stack?

ScalaJVM

How it stacks up

dennybritz/samplercakiki/databasejanikdotzel/akka-http-quickstart-scala
Stars1
LanguageScalaScalaScala
Last pushed2014-05-052026-03-132023-05-19
MaintenanceDormantMaintainedDormant
Setup difficultymoderateeasyeasy
Complexity4/52/52/5
Audienceresearcherresearcherdeveloper

Figures from each repo's GitHub metadata at analysis time.

How do you spin it up?

Difficulty · moderate Time to first run · 1h+

Requires familiarity with the DeepDive data model and preparing tab-separated variable, factor, and weight input files.

The explanation does not mention a license, so usage terms are unknown.

Wtf does this do

This project, called sampler, is a tool for doing probabilistic reasoning over large sets of interconnected data. In plain terms, it helps you figure out the likelihood that certain facts are true, given a web of relationships and rules. It was built to work with a system called DeepDive (developed at Stanford), which is used for tasks like extracting knowledge from text, for example, deciding whether a sentence implies a specific relationship between two companies. At its core, the tool takes three input files: one describing variables (things whose truth you care about), one describing factors (rules or evidence that connect variables), and one with weights that say how much to trust each rule. It then runs a statistical process called Gibbs sampling, which repeatedly estimates the state of each variable by sampling from its neighbors. The result is two output files: one with each variable's final sampled value and its expected probability, and one with learned weights for the rules. The tool also supports "weight learning," meaning it can tune how much each rule matters by running a form of gradient descent over multiple iterations. The main audience would be researchers or engineers working on information extraction, knowledge base construction, or statistical relational learning, particularly those using DeepDive-style pipelines. For example, if you're trying to populate a database of corporate acquisitions from news articles, you might have rules like "if two company names appear near 'acquired' in a sentence, they're likely related." This tool would help compute how confident you should be in each inferred relationship, given all the evidence. One notable tradeoff is performance: the README explicitly says this is not a high-speed implementation and suggests contacting the author if you hit bottlenecks. It's written in Scala and runs on the JVM, which handles threading automatically. The input/output format is straightforward (tab-separated files), making it relatively easy to plug into larger data pipelines, but you'd need to be comfortable with the DeepDive data model to use it effectively.

Yoink these prompts

Prompt 1
I have tab-separated files for variables, factors, and weights from a DeepDive pipeline. How do I run the sampler tool to get inferred probabilities for each variable?
Prompt 2
Help me set up a DeepDive-style pipeline where I extract company acquisition relationships from news articles and use sampler to compute confidence scores for each inferred relationship.
Prompt 3
I want to use weight learning in sampler to tune how much each inference rule matters. How do I configure the weight learning iterations and interpret the learned weights output file?

Frequently asked questions

wtf is sampler?

A Scala tool that uses statistical sampling to estimate how likely certain facts are true, given a set of rules and evidence. Designed for DeepDive pipelines that extract knowledge from text.

What language is sampler written in?

Mainly Scala. The stack also includes Scala, JVM.

Is sampler actively maintained?

Dormant — no commits in 2+ years (last push 2014-05-05).

What license does sampler use?

The explanation does not mention a license, so usage terms are unknown.

How hard is sampler to set up?

Setup difficulty is rated moderate, with roughly 1h+ to a first successful run.

Who is sampler for?

Mainly researcher.

View the repo → Decode another repo

This repo across BitVibe Labs

Don't trust strangers blindly. Verify against the repo.