HACKSPEC_ARTSep 2024
WhiteBox — following an answer back to its sources
By WhiteBox, a few things I'd been playing with started turning into the same problem. Memento had retrieval. I'd also gotten more interested in giving messy information some structure instead of treating everything as plain text.
When a model generates an answer, a lot of the research gets squashed into it. If the answer is wrong, or the part you cared about got dropped along the way, you can't get back to it.
For WhiteBox we took a medical textbook and built it into a graph before anyone asked a question. Concepts linked back to the passages they came from. Different names for the same thing could be merged into one. Relationships the textbook stated outright stayed as relationships in the graph, instead of getting lost in a chunk of text.
Then the chat answer wasn't the end. You could go into the graph and keep working through the material yourself.
I don't want provenance to be a tiny citation badge under a finished answer. I want enough of the work left behind that people can check it, go off in another direction, reuse it somewhere else, or decide the generated answer was wrong in the first place.
The team was me, Kaaustaaub Shankar and Raihan Rafeek, the same crew from Memento and later Gyrus.
| WhiteBox flow |
|---|
| Davidson's medical textbook |
| ↓ |
| Entity / relationship extraction |
| ↓ |
| Clustering + latent representations |
| ↓ |
| Neo4j knowledge graph |
| ↓ |
| User question |
| ↓ |
| MiniLM-assisted retrieval |
| ↓ |
| Relevant graph nodes + source passages |
| ↓ |
| Phi-3 generation |
| ↓ |
| Answer + source / relationship visualization |


In the hackathon version the textbook turned into roughly 60,000 nodes and 130,000 relationships in Neo4j.
A fine-tuned LLaMA 3.1 8B did the summarization/classification during graph construction. MiniLM embeddings supported Cypher retrieval over the graph. Phi-3 3.8B handled the response/deeper-analysis pass on retrieved subgraphs.
Semantic similarity and relationships stated in the source were kept separate. A concept being semantically similar didn't make it a causal or relational edge.
Flask served the backend and vis.js rendered the graph/path visualization.
| Team split | |
|---|---|
| Raihan Rafeek | database and API work |
| Kaaustaaub Shankar | RAG pipeline |
| Arya Garg | open-source model selection, fine-tuning and representation work |
The Devpost wording for my role was "opensource model scout and fine tune monkey!!".12
The database had to be recreated twice, and GitHub went down while we were trying to merge. The internet got bad enough that we physically moved the server to 1819 in a red suitcase.
Neo4j was great for storing the graph and horrible for the interactive visualization we were trying to force on top of it. I remember being really annoyed at that.
We won first place. I knew the idea was cool. Other people thought the idea was cool. I had no clue how to move from cool to okay, who actually pays for this?
After WhiteBox I took a break from serious hackathon building. I was spending more time in NIS and doing much more abstract work than trying to win another weekend.