Skip to content

Add per-minute hero damage, healing and camp stack series #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
geracosta wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Add per-minute hero damage, healing and camp stack series #87

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1794624
Add per-minute hero damage, healing and camp stack series
geracosta Jul 29, 2026
eb968d2
Drop the .mjs changes, keep the diff to the live Java path
geracosta Jul 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 71 additions & 1 deletion src/main/java/opendota/CreateParsedDataBlob.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,9 @@ class PlayerData {
public List<Integer> lh_t = new ArrayList<>();
public List<Integer> dn_t = new ArrayList<>();
public List<Integer> xp_t = new ArrayList<>();
public List<Integer> camps_stacked_t = new ArrayList<>();
public List<Integer> hero_damage_t = new ArrayList<>();
public List<Integer> hero_healing_t = new ArrayList<>();
public List<Entry> obs_log = new ArrayList<>();
public List<Entry> sen_log = new ArrayList<>();
public List<Entry> obs_left_log = new ArrayList<>();
Expand Down Expand Up @@ -142,6 +145,50 @@ public class CreateParsedDataBlob {

private Gson g = new Gson();

// Damage/healing dealt to real (non-illusion) heroes, bucketed by slot and
// minute of the event time, then emitted as cumulative hero_damage_t and
// hero_healing_t series on minute boundaries. Bucketed by event time in a
// pre-scan (rather than accumulated in stream order) because combat log
// entries can appear in the stream after the interval entry of the same
// game time, which would drop the final minute of a match.
private Map<Integer, Map<Integer, Integer>> heroDamageMinuteBySlot = new HashMap<>();

@howardchung howardchung Jul 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to bucket into minutes here? Isn't hero damage/healing additive, so we can just add to it whenever we encounter a combat log event and then add on minute boundaries?

@geracosta geracosta Aug 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Damage is additive — the bucketing isn't about that, it's about stream order. The combat log entries aren't strictly ordered against the interval entries: events for game time T can show up in the stream after the interval entry for T. With a running total snapshotted at each interval, every sample misses the tail of its own minute (it lands one sample late), and anything after the last interval entry is lost for good — that's how I first ran into it, the accumulator version was dropping the final teamfight from the last sample on the match I verified against the scoreboard. Bucketing by event time makes each sample include exactly what happened up to its boundary, regardless of arrival order. Memory-wise it's one int per slot per minute.

@howardchung howardchung Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm if this is true I feel like it's a problem with how we emit the interval events--I think they are supposed to be aligned wiith the game time minutes? And if that's true I think the combat log events should be in the correct buckets.

@geracosta geracosta Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The interval entries are aligned to game time — that part works like you'd expect (onTickStart emits the interval for second T on the first tick where game time reaches T). What's not aligned is where the combat log lands in the stream: intervals are written at tick start, while combat log entries come from the packet messages processed after it, and the message carrying an event can arrive a tick or two after the event's timestamp. So the output stream interleaves records whose time fields aren't globally sorted — a damage record stamped T can sit after the interval record for T. The timestamps are correct; it's the stream position that lags.

Fixing that on the emission side would mean buffering and re-sorting the output by time before writing, with no clean point where it's safe to flush — and it would change stream order for every existing consumer. Even with a perfectly sorted stream, the tail is still lost: intervals stop at postGame, but the fight that ends the game doesn't, so everything stamped after the last interval has nothing later to land in. That's how I ran into this — the accumulator version dropped the entire last teamfight of the match I was checking against the scoreboard.

The existing series don't hit any of this because they sample entity counters (gold/lh/xp are read from PlayerResource state at the tick, so they're consistent at the moment they're read). There's no per-hero damage or healing counter on those entities, so these series have to be summed from combat log events — and once you're summing events, bucketing them by their own timestamp is just using the ground truth they already carry.

private Map<Integer, Map<Integer, Integer>> heroHealingMinuteBySlot = new HashMap<>();
private Map<Integer, Integer> heroDamageCumBySlot = new HashMap<>();
private Map<Integer, Integer> heroHealingCumBySlot = new HashMap<>();

// Events in (60*(k-1), 60*k] belong to bucket k so they are included in the
// sample taken at the minute boundary 60*k
private int minuteBucket(Integer time) {
return Math.max(0, (int) Math.ceil(time / 60.0));
}

private void precomputeMinuteSeries(List<Entry> entries, Metadata meta) {
for (Entry e : entries) {
if (e.time == null || e.value == null) {
continue;
}
// matches the Valve scoreboard definitions: damage/healing dealt to
// real (non-illusion) heroes other than yourself; illusion attacker
// damage counts toward the owning hero
if ("DOTA_COMBATLOG_DAMAGE".equals(e.type) || "DOTA_COMBATLOG_HEAL".equals(e.type)) {
if (e.targethero == null || !e.targethero ||
(e.targetillusion != null && e.targetillusion) ||
e.targetname == null || e.targetname.equals(e.sourcename)) {
continue;
}
Integer sourceSlot = meta.hero_to_slot.get(e.sourcename);
if (sourceSlot == null) {
continue;
}
Map<Integer, Map<Integer, Integer>> target = "DOTA_COMBATLOG_DAMAGE".equals(e.type)
? heroDamageMinuteBySlot
: heroHealingMinuteBySlot;
target.computeIfAbsent(sourceSlot, k -> new HashMap<>())
.merge(minuteBucket(e.time), e.value, Integer::sum);
}
}
}

public ParsedData createParsedDataBlob(List<Entry> entries) {
long tStart = System.currentTimeMillis();
Metadata meta = processMetadata(entries);
Expand Down Expand Up @@ -574,6 +621,7 @@ private boolean hasBuybackBetween(Integer slot, int from, int to) {
private List<Entry> processExpand(List<Entry> entries, Metadata meta) {
List<Entry> output = new ArrayList<>();
precomputeReincarnations(entries, meta);
precomputeMinuteSeries(entries, meta);

for (Entry e : entries) {
String type = e.type;
Expand Down Expand Up @@ -1215,6 +1263,21 @@ private void handleInterval(Entry e, List<Entry> output, Metadata meta) {
addIntervalData(e, output, meta, "xp_t", e.xp);
addIntervalData(e, output, meta, "lh_t", e.lh);
addIntervalData(e, output, meta, "dn_t", e.denies);
if (e.camps_stacked != null) {
// not present in old replays; leave the array empty rather than filling with nulls
addIntervalData(e, output, meta, "camps_stacked_t", e.camps_stacked);
}
int minuteIdx = e.time / 60;
int dmgCum = heroDamageCumBySlot.getOrDefault(e.slot, 0)
+ heroDamageMinuteBySlot.getOrDefault(e.slot, Collections.emptyMap())
.getOrDefault(minuteIdx, 0);
heroDamageCumBySlot.put(e.slot, dmgCum);
addIntervalData(e, output, meta, "hero_damage_t", dmgCum);
int healCum = heroHealingCumBySlot.getOrDefault(e.slot, 0)
+ heroHealingMinuteBySlot.getOrDefault(e.slot, Collections.emptyMap())
.getOrDefault(minuteIdx, 0);
heroHealingCumBySlot.put(e.slot, healCum);
addIntervalData(e, output, meta, "hero_healing_t", healCum);
}
}

Expand Down Expand Up @@ -1476,7 +1539,8 @@ private void handlePosDataTeamfight(Entry e, TeamfightPlayer player) {

// Helper methods
private boolean isArrayField(String type) {
return Arrays.asList("times", "gold_t", "lh_t", "dn_t", "xp_t", "obs_log",
return Arrays.asList("times", "gold_t", "lh_t", "dn_t", "xp_t",
"camps_stacked_t", "hero_damage_t", "hero_healing_t", "obs_log",
"sen_log", "obs_left_log", "sen_left_log", "purchase_log", "kills_log",
"buyback_log", "runes_log", "connection_log", "neutral_tokens_log",
"neutral_item_history").contains(type);
Expand Down Expand Up @@ -1515,6 +1579,12 @@ private List<Integer> getPlayerIntegerList(PlayerData player, String type) {
return player.dn_t;
case "xp_t":
return player.xp_t;
case "camps_stacked_t":
return player.camps_stacked_t;
case "hero_damage_t":
return player.hero_damage_t;
case "hero_healing_t":
return player.hero_healing_t;
default:
throw new RuntimeException("missing list type " + type);
}
Expand Down