Sybil Detection Challenge: Our Modeling Approach
After a little bit of prodding from some friends of ours, we took on the Sybil detection challenge hosted by Human Passport, Octant, and the Ethereum Foundation. Our approach focuses on comprehensive feature extraction from on-chain activity, robust modeling with LightGBM, and heavy use of graph-based and behavioral insights to differentiate between Sybil and human wallets.
1. Feature Engineering Across Chains
We started by building rich behavioral profiles for each wallet using data from both Ethereum and Base chains. This included transactions, token transfers, and DEX swaps.
We engineered features in the following categories:
- Basic Activity Metrics: Counts of sent and received transactions and token transfers, as well as the diversity of counterparties interacted with.
- Monetary Behavior: Aggregated ETH and token values, standard deviations, and fee ratios.
- Temporal Cadence: Wallet activity timelines such as first/last seen days, active days, inter-transaction timing, and weekday/hour entropy.
- Gas Usage: Statistical summaries of gas usage and low-gas behavior flags.
- Swap Behavior: DEX activity including number of swaps, swap pair diversity, and USD volumes.
- Ping-Pong Patterns: Detection of rapid back-and-forth transaction motifs within small time windows.
- Graph Embeddings: We built an undirected graph using all transaction, transfer, and swap edges, and used Node2Vec to embed each wallet into a 128-dimensional vector space capturing its structural role in the network.
We applied all of the above feature extractions individually to Ethereum and Base chains. Then we created ratio features comparing Ethereum to Base behavior to detect inconsistencies that are typical in Sybil activity.
2. Graph Embedding with Node2Vec
The embedded transaction graph helped capture wallet position and behavior in the broader network. We used Node2Vec with tuned parameters (walk length, context window, etc.) to generate dense vector embeddings for each address. These embeddings were merged with engineered features to form the final feature matrix.
3. Data Preparation and Augmentation
We combined labeled addresses from both Ethereum and Base chains, removing duplicates to create our training set. We also integrated the Ben2k list of suspected false negatives by relabeling them as Sybil addresses.
To combat class imbalance (about 1:38 Sybil:human), we applied undersampling to the majority class during training.
4. Model Training and Optimization
We used LightGBM for model training, which offered a good balance between speed, interpretability, and performance. Our pipeline included:
- Stratified 7-fold cross-validation
- Optuna-based hyperparameter tuning per fold (40 trials)
- Fold-wise out-of-fold predictions and AUC reporting
- Feature importance tracking for gain and split
After evaluating CV performance (OOF AUC), we retrained on the full dataset using the median of best-performing hyperparameters and average best iteration count.
5. Prediction and Submission
The final model was used to generate probability scores for all test addresses. We ensured full coverage of the test set and saved the results as a compliant CSV for submission.
6. Final Thoughts
This competition challenged us to deeply understand behavioral and structural signals of Sybil activity. Our approach emphasized a multi-layered analysis: raw transactional patterns, temporal dynamics, network structure, and cross-chain inconsistencies. We’re proud of the system we built and look forward to seeing its impact in ongoing Sybil resistance efforts.
Appendix: The Feature Buffet ! 
Below is a richer tour of the ten feature families we baked into the model. Think of each family as a camera angle on wallet behavior; together they create a full-body X-ray that exposes the cardboard cut-outs hiding among real humans.
- Activity Volume – “How much noise do you make?”
-
(Transactions and Transfers) A Sybil farmer usually spins dozens of low-stake wallets that do something just often enough to scrape eligibility rules. Counting how many times an address pushes (or receives) ETH or tokens lets us spot:
- Under-active ghosts – nearly empty wallets created only for claim day.
- Over-active bots – conveyor-belt addresses that fire hundreds of micro-tx per hour.
-
(Unique tokens touched) Legit users’ holdings tend to reflect personal taste, while Sybils favor whatever asset qualifies for a campaign. A wallet that sends and receives 32 different obscure airdrop tokens but never stable-coins? Suspicious.
- Monetary Footprint – “Show me the money”
-
(ETH statistics (sum, mean, median, max)) High-balance whales and zero-balance zombies behave very differently; both extremes are easy to flag. We also compute
- Coefficient of variation – Sybils often repeat the exact value (e.g., 0.005 ETH) to satisfy “non-zero” filters without overspending, yielding almost-zero variance.
-
(Fee ratio (gas fees vs. value sent)) Humans hate overpaying. If a wallet routinely burns fees that dwarf the value being moved, it might be an automated faucet script that doesn’t care about ROI.
-
(USD volume on tokens) Translating token amounts into USD normalises across meme-coins and blue-chips, exposing wallets that shovel large volumes of valueless tokens just to look busy.
- Temporal Rhythm – “When do you show up?”
-
(First seen / Last seen / Lifetime) A wallet born yesterday and already hyper-active screams throwaway burner.
-
(Active-days count) Real users visit sporadically over months; Sybils often crunch everything into a single weekend.
-
(Mean & standard-deviation of inter-tx seconds + Burst-iness z-score) Bots fire in tight, consistent clusters. Humans are messy: morning coffee, lunch break, midnight DeFi spree. Extreme regularity is a red flag.
-
(Weekday & hour entropy) Entropy near zero ⇒ all actions happen on one weekday/hour (scripted). High entropy ⇒ spread across the calendar (organic).
-
(Night-owl ratio (00-05 h UTC)) Legit Western users sleep; Sybil scripts happily continue. A sky-high night ratio is incriminating.
- Gas-Price Habits – “Do you pinch gwei?”
-
(Mean / median / std gas price + Coefficient of variation) Batch scripts often reuse fixed gas parameters; variance collapses.
-
(Low-gas counter (bottom 10 % by block)) Farmers queue transactions at absurdly low prices to save pennies; patient humans generally pay market rate to avoid stuck txs.
- Counter-Party Diversity – “Who are your friends?”
- Unique addresses contacted (outgoing) and unique sources (incoming)
- Tight bubble: one wallet whips tokens between the same 3 siblings → Sybil farm.
- Wide circle: dozens of unrelated peers → natural trading or DeFi use.
- Swap Behaviour – “Do you actually trade?”
- Swap count - Airdrop-only wallets often skip DEXs completely; professional farmers do one obligatory swap of tiny size.
- Pair diversity - Engaged DeFi users explore many markets (ETH/USDC, wBTC/ETH, etc.). Low diversity means checklist activity.
- Total USD swapped - If the dollar volume is tiny but fees are paid anyway, motive is probably eligibility, not profit.
- Ping-Pong Flag – “Are you bouncing tokens with yourself?”
We scan for back-and-forth transfers between two addresses inside a 10-minute window. That self-wash is textbook Sybil: inflate “number of engaged wallets” metrics without spending real money.
- Social Graph Embeddings – “Who sits near you in the giant address graph?”
All tx, transfers, and swaps across both chains are stitched into a single undirected network. Node2Vec then wanders these edges to learn a 128-dim vector for every node—capturing:
- Closeness to popular hubs (DEX routers, bridges).
- Membership in dense farmer clusters (many edges but low entropy).
- Isolation (bridges only to its creator’s main wallet).
These embeddings often reveal structure the naked eye misses.
- Cross-Chain Personality – “Are you a chameleon?”
For every Ethereum feature that also exists on Base we compute eth / base ratios. Genuine users often behave similarly across chains (same bedtime, same gas strategy). A ratio that explodes—for example, 100× more swaps on Base than on ETH—may indicate a wallet spun up solely to farm a Base campaign.
- Meta-Flags & Hygiene
- Missing-value masks – sometimes “feature = 0” is informative (e.g., never swaps).
- Infinity guards & universal fill-0 – keeps LightGBM happy and prevents freak explosions in ratio features.