Hack the Video Agent Context Graph

TransactionShield is a safe financial-transaction security review demo. Auth0 provides authentication, MFA, session context, and reviewer roles. Stripe simulates PaymentIntents and webhook-style transaction status updates. TwelveLabs and Neo4j provide consented video evidence and relationship context for abnormal transactions. A human reviewer makes every final decision; no real money moves and no automatic denial occur

TwelveLabsNeo4jOpenAI

supercut agents

A knowledge graph over indexed video. Ask a relational question and get back one stitched clip of exactly the moments that answer it — plus the traversal that found them.

TwelveLabsNeo4jOpenAI

Log various expressions that foo demonstrates in these 2 videos and count the duration foo's expressions are visible. Could not complete within the timeframe, so just submitting the idea now.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

hack2hacks

watches recordings of San Francisco government meetings and turns them into something you can actually search and ask questions about. Instead of sitting through hours of video, you can ask things like “who said this project was approved?” or “has this person changed their story?” When someone claims a project was greenlit, the app also checks the city’s public building-permit records to see if that matches what’s really on file. In short: it helps people hold public statements up against the public record — without watching every meeting.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

Framewise

A Product that turns the video into reusable intelligence with memory - for Sales & Makering videos - Security Vigilance. Designed with scaled in mind.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

Wake is an AI Rowing Coach that uses Concept2 log data and TwelveLabs Pegasus and Jockey to derive technique and form insights. It's all wired together into Neo4j Aura into an evidence ontology. Then OpenAI is driven through Strand to navigate the graph to create insights that lead to future technique drills.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

Hard Drive

Most video QA pipelines process the video, create a knowledge graph and then answer the questions based on that graph. We invert this idea - instead of building the graph from the video, we build it from the questions. When a question arrives, the agent takes a fast, low-cost glance at the video, streams a provisional answer with a confidence estimate in seconds, and then chooses the single most informative next perception call. It asks which minute to have Pegasus describe closely, which probe to run through Marengo. Then it updates its belief, and repeats until its confidence in its answer clears a threshold or a (price) budget runs out. Every fact it learns along the way is accumulated (memoized) into Neo4j with evidence and cost attached, so the graph is the accumulation of what users care about.

TwelveLabsNeo4jStrands AgentsOpenAIAnthropic API

Beeline

Type a concept. Beeline resolves it to a node in the graph, computes its full prerequisite closure, prunes everything you tick as already known, selects the minimum set of clips that genuinely teaches what remains, orders them so prerequisites always play first, and plays them as one continuous session.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

SemanticDataLabs

tag line: automated task-aware semantic annotation + search for research media Description: AI/ML researchers/labs working on massive video/audio datasets often spend high amounts for data annotations to understand what is in the data. Often this needs to be redone for different tasks and even then there is human error possibilities. Existing metadata is often not detailed to second or event level for unstructured data like audio/video. Semantic Data Labs provides an automated data annotations, metadata extractions and quick subset search and filtering to quickly get started on ML tasks. How it works: the 5-step pipeline (ingest → TwelveLabs extract + annotations with LLM → OpenAI interpret → graph index → search/export) with the architecture diagram What's distinctive: task-aware annotation, versioned iterations, graph-native temporal search, zero-config local run, separate mock/live corpora What's built: web dashboard for users to upload datasets, generate annotations and metadata, and search for subsets.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

Supercut

Every clipping tool sees one video at a time. Supercut watches the whole channel—and turns its history into one coherent edit. A creator uploads multiple videos and gives Archie, our AI director, a brief such as: “Every time Joma flexes money, from small to insane.” TwelveLabs analyzes the videos across vision, speech, audio, and on-screen content, producing timestamped moments. Strands Agents orchestrates the pipeline while OpenAI identifies entities, recurring bits, numerical values, and cross-video relationships such as ESCALATES and CALLBACK_TO. Neo4j stores those moments and relationships as a context graph. The Director converts the brief into a graph query and an ordered edit decision list. In the interface, the selected path lights up across the graph, every proposed cut includes a rationale, and the editor can veto individual moments before rendering. FFmpeg then produces a music-backed 720p montage. Cached, uniformly encoded segments make revisions fast to re-stitch. The key idea is that the edit is not an opaque AI answer or a list of search results. It is an explainable path through the creator’s history. Every cut exists because a relationship exists in the graph.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

SiteTrace

SiteTrace is an AI-powered safety investigation system for construction sites. It reconstructs incidents and near-misses from multi-camera footage, compares the observed work against the approved Job Hazard Analysis (JHA), and generates evidence-backed safety documentation. The core workflow combines: - TwelveLabs for video understanding, segmentation, search, and clip-level citations - OpenAI for reasoning over JHA steps, observed actions, and safety deviations - Neo4j for modeling planned vs. observed work as graphs and computing differences - Strands Agents for orchestrating the end-to-end investigation pipeline - AWS for storage, async processing, and deployment The result is not a generic AI summary. It produces the same artifacts a safety manager would normally create by hand: - JHA Compliance Review - Safety Observation - Corrective Action Plan - Optional issue handoff for tools like Autodesk Build or Procore SiteTrace helps teams move faster from footage to verified safety action, while keeping every claim tied to evidence.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

J

J

Team 68

An AI-assisted tool that helps linguists study how Mandarin sentence-final particles work with tone and gestures. It analyzes videos, suggests transcripts and gestures, and lets researchers review everything before saving confirmed results.

TwelveLabsNeo4jOpenAI

AgentHackers

Extract intelligence from police bodycams and answer questions related to cases.

TwelveLabsNeo4jStrands AgentsOpenAI

RushHour2

Puffer AI analyzes full-length livestreams to identify viral moments by understanding faces, dialogue, emotion, audience reactions, callbacks, and narrative context. It delivers ranked clip timestamps, explains why each moment could spread, and teaches creators how to recreate the winning mechanic in their own style.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

SceneThread

SceneThread transforms fragmented CCTV and surveillance footage into an intelligent, searchable source of real-world context. It solves a critical problem: important people, objects, actions, and incidents remain buried across hours of security-camera video, making investigations slow and manual. TwelveLabs analyzes multimodal footage and retrieves precise moments, while OpenAI converts that evidence into structured insights and grounded answers. Neo4j connects scenes, entities, events, and timelines into a persistent context graph, revealing relationships across the surveillance video. Strands Agents efficiently orchestrate the end-to-end workflow while maintaining traceability, confidence, and timestamped evidence. The result is an explainable, conversational CCTV intelligence system that helps teams find and understand critical incidents in seconds.

TwelveLabsNeo4jStrands AgentsOpenAI

P

Rehearsal

Rehearsal is an AI public-speaking coach. Record a practice talk on camera — we transcribe it LIVE, score your delivery (pace, fillers, clarity, structure, confidence) using OpenAI and Twelve Labs, and generate a timestamped debrief. Every strength and weakness links to a coaching segment from a world-class talk (MIT, TEDx, Winston, Chris Voss, Stanford GSB, etc) to watch — clips retrieved with TwelveLabs and mapped through a Neo4j knowledge graph of speaking techniques. Strands Agents orchestrates the reasoning and allows for an independent exploration of speaking techniques.

TwelveLabsNeo4jStrands AgentsAWSOpenAINext.js, MediaRecorder

Agents know what we say, but miss what happened between the lines. Onari Observed Reflection is a Swift iPhone app that captures voice, front-camera video, or a user-started observer session as a time-grounded day-context stream. TwelveLabs finds multimodal moments; Strands Agents orchestrate an OpenAI reading and counter-reading against one redacted Onari context anchor; Neo4j persists the evidence paths. In History, the user can return to a reflection, rewind what was seen, inspect why it shaped the reading, correct it, or delete it—turning observed signals into inspectable personal memory without presenting observation as diagnosis.

TwelveLabsNeo4jStrands AgentsOpenAISwiftUI, ReplayKit, FastAPI, Python, Onari Bridge

Idea Genome AI watches technical talks and finds connections between them that nobody pointed out - a problem raised in one video that's already solved in another, ideas that keep showing up across multiple talks, and problems nobody has proposed a solution for yet. It uses TwelveLabs to understand what's shown, said, and written in each video, Strands Agents and OpenAI to extract and reason over that content, and Neo4j to store it all as a connected knowledge graph. Instead of waiting for a question, it proactively surfaces what it found - and every finding links to the exact timestamp in the source video, so you can verify it instead of just trusting the AI.

TwelveLabsNeo4jStrands AgentsOpenAIReact, FastAPI, Neo4j AuraDB, dagre/React Flow

karpathy_graph

The problem Video is where ideas go to hide. Andreji Karpathy has given about a hundred hours of talks. Nobody has watched them all, and the interesting questions never live in one video. How did his thinking on agents change? Where does 2025-him push back on 2023-him? Search finds keywords. It never finds a change of mind. Before: hours of footage nobody watches, answers scattered across years, quotes traveling without their date or context. After: every claim extracted, dated, and linked to its exact clip. One question reaches every talk at once. Contradictions surface themselves as graph edges. Every answer plays the receipt.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

SHARE.io

A webapp that takes an video and also the tone to generate captions, story to post on LinkedIn This is not an app for filling the gap on creative thinking of people but aiding people to share their thoughts who are hesitant to do otherwise like me.

TwelveLabsAWSOpenAI

cat-neko-goyangi

Whisker is a tiny wearable audio-video device that takes notes on everyone and everything you've seen. It compiles everything through a Neo4j graph and then gives a summary of the day, who you met, and action items, with the ability to look things up after the fact.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

MealPrep

Transform cooking videos into interactive knowledge graphs. Understand recipes, nutritional claims, and meal prep strategies through AI-powered video analysis.

TwelveLabsNeo4jStrands AgentsOpenAI

Video sources contradict each other and nobody checks. The Unreliable Narrator ingests videos into a context graph, then answers the questions a transcript search can't: how many videos claim X, what did no video mention, and which two videos disagree — with both clips cued up to the second. TwelveLabs returns what was spoken, what was on screen, and what was visible as three separate signals. We keep them separate all the way into Neo4j, so a claim stays attached to the scene, the speaker, and the timestamp it came from. OpenAI types the entities and adjudicates disputed claims against the literature. Strands drives the read path over the graph. The result is a graph you can ask negative and comparative questions of. Entity search tells you a video mentions creatine. This tells you seven videos claim a dose, two of them contradict each other, and here are the four seconds where each one says it.

TwelveLabsNeo4jAWSOpenAIStrands Agents SDK (AWS) for the agent read path · TwelveLabs Analyze for multimodal segment extraction · OpenAI GPT-5.6 (luna for scene enrichment, terra via OpenAIResponsesModel for adjudication) · Neo4j Aura for the context graph · FastAPI backend · React + Vite + D3-force for the graph UI · yt-dlp + ffmpeg for clip extraction · Playwright for e2e

ZooVision

ZooVision is a night time animal monitoring system for zoo and sanctuary teams. It turns recorded enclosure video into timestamped behavior observations, compares those observations with each animal's own daytime baseline, and sends staff evidence-backed welfare-check prompts when deterministic rules find something unusual.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

Delta-Learning

Delta Learning is a personalized learning buddy that turns any video into a study experience built around the student. It ingests videos, identifies timecoded concepts with TwelveLabs, and connects them in Neo4j to the student’s notes, demonstrated knowledge, and learning goals. OpenAI and Strands Agents then create “Your Cut”: the exact segments the student should watch, what they can safely skip, and why each recommendation matters. Afterward, Delta helps the student study with targeted questions and records only the concepts they can demonstrate—not merely what they watched. As their knowledge graph grows, future recommendations become more focused, cut lists become shorter, and learning plans adapt to what they actually need next. If the existing video library cannot fully support a goal, Delta can discover and ingest new material—giving every student a learning buddy that understands what they know, helps them study, and grows alongside them.

TwelveLabsNeo4jStrands AgentsOpenAI

Patient Zero AI Finding exposure before it becomes an outbreak. During the Nipah outbreak in Kerala, India, infection-control teams spent days manually reviewing surveillance footage trying to figure out who was exposed. I asked one question: what if AI could do that in minutes? Patient Zero AI is an investigation platform. You upload surveillance videos. The AI watches them, understands what happened, and helps you find who was exposed before the disease spreads further. How it works You upload one or more surveillance videos from a hospital, clinic, or public space. The system analyzes every frame using TwelveLabs, which extracts people, their descriptions, how long they were near each other, and what happened. That analysis goes to OpenAI which reasons across the footage and builds an investigation summary. You then chat naturally with the AI to identify the prime suspect and everyone around them. When you are ready, you generate an exposure graph showing Patient Zero at the top and every exposed person below, color coded by risk. Red means quarantine required. Amber means monitor closely. Green means out of danger. The watch list then ranks everyone by how much time they spent near Patient Zero and how close they were. The entire pipeline is orchestrated by AWS Strands Agents, stored in Neo4j as a knowledge graph, and reasoned over by OpenAI GPT-4o. This is not a diagnosis tool. It is an investigation assistant that gives infection-control teams a head start when every hour matters. Tech Stack TwelveLabs for multimodal video understanding OpenAI GPT-4o for cross-video reasoning and natural language chat AWS Strands Agents for workflow orchestration Neo4j AuraDB for knowledge graph storage Next.js 15 for the full stack application

TwelveLabsNeo4jStrands AgentsAWSOpenAI

A multimodal match-analysis system that converts Street Fighter 6 tournament footage into a temporal context graph of tactical sequences (pressure, projectile exchanges, drive mechanics, knockdowns, neutral, combos), player decisions, outcomes, and derived tendencies. TwelveLabs Pegasus segments and classifies tactical episodes; Marengo transcription supplies commentary evidence; Neo4j chains and grounds them; Strands agents orchestrate both paths — an ingestion agent that triages perception quality and decides what is trustworthy enough to load, and an analyst agent that answers tactical questions with evidence citations and honest uncertainty.

TwelveLabsNeo4jStrands AgentsOpenAI

Construction Sight

Construction Sight provides visual intelligence for digital twins. Digital Twins require enormous coordination, logistics, and data acquisition. When a process for collecting useful information is high-friction, it typically doesn't get done. Video and audio voiceover are occasionally collected on-site to document major issues, but the high-friction workflow prevents rapid, frequent capture of observations, emerging concerns, and other ongoing issues. Compounding this, a site's relevance doesn't end when construction does. A digital twin needs to maintain context indefinitely, across the full lifespan of the site. Construction Sight helps close this gap. By providing a low-friction video catalog with a semantic backing, it lets operators quickly take notes, scan the site, or capture broader site coverage, since the cost of acquiring visual documentation drops significantly. It also interoperates with 3D reconstruction workflows: 3D Gaussian Splats (3DGS) are often generated from video, and Construction Site adds operable semantic meaning on top of that footage. For this demo, we sourced input scan videos from pre-existing 3DGS captures — a more cost-effective alternative to an on-site visit.

TwelveLabsNeo4jStrands AgentsAWSOpenAICesium

Machanize

CaseGraph turns videos, images, and interviews into a searchable investigation graph. It creates evidence timelines, connects people, objects and events, checks suspect claims for support or contradictions, and generates investigator briefs where every conclusion links to its source and exact timestamp.

TwelveLabsNeo4jStrands AgentsAWSOpenAI

A video agent that watches the news about a borrower and tells you which story actually threatens the loan — and which one just looks like it does. Big loans carry big promises: keep earnings above a set multiple of the interest bill, keep debt below a set share of asset value. Break one and the lenders can act. Material Witness watches the news about a borrower's infrastructure and works out which story actually threatens those promises — and which is just loud. TwelveLabs indexes the footage; a Neo4j graph traces every claim back to the second of video it came from. The agent may only propose; a human signs before anything counts. The test case is Gatwick Airport. Drones shut it for 33 hours in 2018 and led every news bulletin on earth — the debt never noticed. The stoppage of air travel in 2020 was covered far more quietly and forced its lenders to suspend the loan's terms. Across six broadcasters the system found 106 links to financial concepts and proved none legitimately connects to the loan terms. Reporting that gap is the product.

TwelveLabsNeo4jStrands AgentsAWSOpenAIRender, You.com

EarlyWarning

Upsert workplace camera video and autonomous machinery footage to identify 'near-misses' in machine and equipment handling by people and AI that could cause real injuries and real lives, which is analyzed to prevent the near-miss from turning into a future real accident with actionable insights by OpenAI.

TwelveLabsNeo4jStrands AgentsAWSOpenAIVercel for frontend deployment, check it out! https://earlywarnings.vercel.app/

Startup Pitch Network watches YC Demo Day pitch videos and turns them into a Neo4j knowledge graph an agent can reason over. We ingested 62 pitches spanning ten years of Demo Days (S16 to S26). For each video, TwelveLabs Pegasus reads the slides and transcribes the speech, and OpenAI structures the result into typed nodes: startups, founders, problems, technologies, industries, and every traction claim with the second it was said on stage. A second pass clusters problems that are worded differently but mean the same thing, then derives competitor edges from the clusters. Neo4j stores the graph, and a Strands agent answers questions over it with Cypher and vector search, lighting up the relevant subgraph in a live visualization. Ask "which startups are competing and on what?" and it walks 135 derived competitor edges. Ask "list Basis's traction claims" and it cites "4 signed contracts" at 1:00, matching the slide on screen at that second. Ask "how did what founders build change after ChatGPT?" and it answers from the graph: AI moved from a backend feature in 2019-2021 (Embrace, Malloc) to the product itself in 2023-2026 (Sync, Contrario, Magic Loops). Every fact links back to a frame and a timestamp, and the cross-video and trend queries have no embedding-search equivalent. Built on the video-context-graph starter; our additions are the pitch ontology, the clustering pass, the batch-over-batch trend layer, and parallel ingestion.

TwelveLabsNeo4jStrands AgentsOpenAIFastAPI, Next.js + Neo4j NVL, Docker, yt-dlp, Python/uv

Composa

Composa is a video editor on the mental model of layer based editing tools like Canva and Figma. Think of adding a Figma design on top of a video. Designers and marketers can use it to create branded video since it feels a familiar tool (same mental model plus they can copy-paste from Figma) For this hackathon I added video understanding so the editor's agent can work from what is actually in the video footage. TwelveLabs indexes the video, a Strands agent queries a Neo4j graph of segments and speech, and the agent edits the timeline directly: placing clips at the in and out points it chose, and timing animations to the moment something is said. This creates an experience where an agent and a human could edit the video together. The agent can place clips and add animations and you can undo or edit its changes. The app uses an allowlist to prevent the agent from dangerous commands like deleting your layers or your composition. My video is one take, first take, no cuts. Please bear with me haha.

TwelveLabsNeo4jStrands AgentsAWSOpenAIClaude Code, Replicated UI3 Design System

Prampta

Current AI video generators have a massive loophole. If a user prompts for a protected subject by name, the generation is blocked by standard text filters. However, if the user inputs a highly detailed physical description of that person without using their name, the system happily generates the video.

TwelveLabsOpenAI

Nightingale

Nightingale is an AI-powered patient safety platform for continuous, personalized monitoring as patients move from intensive care to a general hospital ward. Using computer vision and multimodal video analysis, it establishes a patient-specific behavioral baseline—including movement, bed activity, speech, and daily routines—while the patient is closely monitored in the ICU. After transfer, Nightingale compares new activity against that baseline to identify meaningful deviations. A local, real-time safety system detects urgent events without relying on the cloud. TwelveLabs interprets patient video, Neo4j maintains an auditable behavioral history, OpenAI generates evidence-backed explanations, and Strands Agents orchestrates the multimodal analysis and escalation workflow. Actionable alerts reach nurses through a pager or Twilio call, supporting faster prioritization without presenting the system as a diagnostic tool.

TwelveLabsNeo4jStrands AgentsAWSOpenAITwilio

RightCall

This is a video assistant agent that will help referees make right calls under pressure and during peak situations. we all know how important every call is.

TwelveLabsNeo4jStrands AgentsOpenAI

mn mn

TwelveLabsNeo4jAWSOpenAI

The best team.

The detailed description, but not overly detailed. If you give us a big chunk of AI-generated paragraph text, the humans that are going to review this are just going to skip over it.

TwelveLabsNeo4jAWSOpenAI

test

Neo4j