Discord Chat Analyzer

GitHub Repository

The DCA operates on static JSON chat exports, downloaded by the DiscordChatExporter.

Privacy and authorization

Chat exports can contain messages, usernames, IDs, attachments, and other personal data. Analyze only data you are authorized to access, keep the exports out of version control, and remove or anonymize personal data before sharing a report. Do not expose Discord account tokens to this tool or commit them to scripts.

Architecture and Output

After parsing all .json logs into Java entities, each Channel is analyzed by Analyzer.java. The analyzer reads each message and tracks a multitude of information for each specific author/chatter, automatically sorted by TreeMaps.

Place the downloaded .json files in a local directory outside the repository, and update the input directory in readLogs accordingly. Keep private exports out of the repository.

Based on the rankingType, the analyzer writes its data to separate files.

RankingType.java

package analyzer.models.ranking;

public enum RankingType {
    MOST_MESSAGES,
    MOST_EMBEDS,
    MOST_ATTACHMENTS,
    TIMES_MENTIONED,
    ACCOUNT_AGE,
    MOST_COMMON_REACTION,
    AVG_WORD_COUNT
}

ACCOUNT_AGE, calculateAccountAgeRanking, and joinedServer are legacy implementation names in these examples. They refer to the earliest message for each author in the export, not account age or server-join date.

The data is aggregated and analyzed by an implementation of Ranking.java.

Ranking.java

import analyzer.stats.AuthorData;
import lombok.Getter;
import lombok.Setter;

import java.util.List;

public abstract class Ranking {

    @Getter
    @Setter
    private transient List<AuthorData> authorDataList;

    public Ranking(List<AuthorData> authorDataList) {
        this.authorDataList = authorDataList;
    }

    public String getOutputFilePath() {
        return "logs/not-implemented.json";
    }
}

Ranking implementations

Program Sequence

Entity Models

By manually analyzing the available fields in the json logs, a separate pojo / entity file has been created for each object. Using Lombok, the Pojos can be kept fairly small. This could be reduced even further, by implementing each entity as a Java 17 record.

Each server (Guild) has multiple Channels. Each channel has ChannelInfos and n messages.

Channel.java

import analyzer.models.DateRange;
import analyzer.models.Guild;
import analyzer.models.message.Message;
import lombok.Getter;
import lombok.Setter;

@Getter
@Setter
public class Channel {

    private Guild guild;
    private ChannelInfo channel;
    private DateRange dateRange;
    private Message[] messages;
}

ChannelInfo.java

import lombok.Getter;
import lombok.Setter;

@Getter
@Setter
public class ChannelInfo {
    private String id;
    private String type;
    private String categoryId;
    private String category;
    private String name;
    private String topic;
}

Message.java

import analyzer.models.Author;
import analyzer.models.message.embed.Embed;
import analyzer.models.message.reaction.Reaction;
import lombok.Getter;
import lombok.Setter;

@Getter
@Setter
public class Message {
    private String id;
    private String type;
    private String timestamp;
    private String timestampEdited;
    private String callEndedTimestamp;
    private boolean isPinned;
    private String content;
    private Author author;
    private Attachment[] attachments;
    private Embed[] embeds;
    private Reaction[] reactions;
    private Mention[] mentions;
    private Reference reference;
}

Load JSON

We instantiate a Gson object and read all json log files in parallel.

Main.java

private static List<Channel> parseJsonToChannels() {
        final Gson gson = new GsonBuilder().setDateFormat(DateFormat.FULL, DateFormat.FULL).create();

        final List<Channel> channels = Collections.synchronizedList(new ArrayList<>());
        final List<String> logs = readLogs();

        logs.parallelStream().forEach(logFilePath -> {
            try (Reader reader = Files.newBufferedReader(Paths.get(logFilePath))) {
                final Channel channel = gson.fromJson(reader, Channel.class);
                channels.add(channel);
            } catch (Exception ex) {
                ex.printStackTrace();
            }
        });

        return channels;
    }

Data Aggregation

The loaded log data gets send through multiple methods of data aggregation and preparation. For each single author / chatter found.

populateAuthorDataMap()

private void populateAuthorDataMap(AuthorData authorData, Message message) {
        analyzeMessage(authorData, message);
        authorData.setAuthorId(message.getAuthor().getId());
        authorData.setAuthor(message.getAuthor());
        authorData.setEarliestLocalDate(message.getTimestamp());
        authorDataMap.put(message.getAuthor(), authorData);
}

analyzeMessage()

private void analyzeMessage(AuthorData authorData, Message message) {
        authorData.incrementMessages();
        analyzeContent(authorData, message);
        analyzeEmbeds(authorData, message);
        analyzeAttachments(authorData, message);
        analyzeMentions(authorData, message);
        analyzeReactions(authorData, message.getReactions());
}

The analyzeMentions method for example counts the amount of times, the current chatter has been mentioned (e.g. @UserXYZ).

analyzeMentions()

private void analyzeMentions(AuthorData authorData, Message message) {
        final Mention[] mentions = message.getMentions();
        if (mentions != null && mentions.length > 0) {
            authorData.incrementTimesMentioned();
        }
}

Each Author collects its own data, which get updated during the analyzation process.

AuthorData.java

@Getter
@Setter
public class AuthorData {

    private transient Author author;
    private transient LocalDate earliestLocalDate;
    private transient List<Integer> wordsPerMessage = new ArrayList<>();

    private String authorId;
    private String firstMessageSent;
    private double averageWordsPerMessage;
    private long messagesSent = 0;
    private long embedsSent = 0;
    private long attachmentsSent = 0;
    private long sumEmojisReceived = 0;
    private long timesMentioned = 0;
    private Map<Emoji, Integer> emojisReceived = new TreeMap<>(new Emoji.EmojiComparator());

// [...]
}

Statistics Generation

Each data entry gets fed to a Java TreeMap which integrates a custom comparator. The TreeMap is then automatically sorted according to the compare method written. After all data has been ingested, the resulting TreeMap is already in the correct ranked order.

For example, the author, who has been mentioned the most, is on index 0.

Analyzer.java

@Nullable
public Ranking getRanking(RankingType rankingType) {
    Ranking result = null;

    switch (rankingType) {
        // [...]
        case TIMES_MENTIONED:
            result = new TimesMentionedRanking(new LinkedList<>(authorDataMap.values()));
            break;
    }

    return result;
}

calculateMentionRanking()

private void calculateMentionRanking(List<AuthorData> authorDataList) {
    timesMentioned = new TreeMap<>(new AuthorData.AuthorDataMentionsCountComparator());
    authorDataList.forEach(authorData -> timesMentioned.put(authorData, authorData.getTimesMentioned()));
}

Earliest message in the export

Author with the earliest message represented in the exported data.

This is not the account's creation date or necessarily the date they joined the server. It is limited by the exported channels and the available history.

Comparator

public static class AuthorDataFirstMessageComparator implements Comparator<AuthorData> {
    @Override
    public int compare(AuthorData o1, AuthorData o2) {
        final int compare = o1.getEarliestLocalDate().compareTo(o2.getEarliestLocalDate());
        if (compare == 0) {
            return 1;
        } else {
            return compare;
        }
    }
}

Analysis

private void calculateAccountAgeRanking(List<AuthorData> authorDataList) {
    joinedServer = new TreeMap<>(new AuthorData.AuthorDataFirstMessageComparator());
    authorDataList.stream()
            .filter(authorData -> authorData.getMessagesSent() >= 10)
            .forEach(authorData -> joinedServer.put(authorData, authorData.getLocalDateAsString(authorData.getEarliestLocalDate())));
}

Average Words per Message

Comparator

public static class AvgWordCountComparator implements Comparator<Double> {
    @Override
    public int compare(Double o1, Double o2) {
        final int compare = o2.compareTo(o1);

        if (compare == 0) {
            return 1;
        } else {
            return compare;
        }
    }
}

Analysis

private void calculateAvgWordCount(List<AuthorData> authorDataList) {
    authorDataList
            .stream()
            .filter(authorData -> authorData.getMessagesSent() >= 10)
            .forEach(authorData -> {
                final double wordCountSum = authorData.getWordCountSum();
                final double messagesSent = authorData.getMessagesSent();

                if (wordCountSum > 0 && messagesSent > 0) {
                    authorData.setAverageWordsPerMessage(round(wordCountSum / messagesSent));
                    averageWordsPerMessage.put(authorData.getAverageWordsPerMessage(), authorData.getAuthor().getNickname());
                }
            });
}

Attachments added

Comparator

public static class AuthorDataAttachmentsCountComparator implements Comparator<AuthorData> {
    @Override
    public int compare(AuthorData o1, AuthorData o2) {
        final Long attachmentsSent1 = o1.getAttachmentsSent();
        final Long attachmentsSent2 = o2.getAttachmentsSent();
        final int compare = attachmentsSent2.compareTo(attachmentsSent1);

        if (compare == 0) {
            return 1;
        } else {
            return compare;
        }
    }
}

Analysis

private void calculateAttachments(List<AuthorData> authorDataList) {
        mostAttachments = new TreeMap<>(new AuthorData.AuthorDataAttachmentsCountComparator());
        authorDataList.forEach(authorData -> mostAttachments.put(authorData, authorData.getAttachmentsSent()));
}

private void countAttachments(List<AuthorData> authorDataList) {
    for (AuthorData authorData : authorDataList) {
        attachmentsSent = attachmentsSent + authorData.getAttachmentsSent();
    }
}

Embeds added

Comparator

public static class AuthorDataEmbedsCountComparator implements Comparator<AuthorData> {
    @Override
    public int compare(AuthorData o1, AuthorData o2) {
        final Long embedsSent1 = o1.getEmbedsSent();
        final Long embedsSent2 = o2.getEmbedsSent();
        final int compare = embedsSent2.compareTo(embedsSent1);

        if (compare == 0) {
            return 1;
        } else {
            return compare;
        }
    }
}

Analysis

private void calculateEmbeds(List<AuthorData> authorDataList) {
    mostEmbeds = new TreeMap<>(new AuthorData.AuthorDataEmbedsCountComparator());
    authorDataList.forEach(authorData -> mostEmbeds.put(authorData, authorData.getEmbedsSent()));
}

private void countEmbeds(List<AuthorData> authorDataList) {
    for (AuthorData authorData : authorDataList) {
        embedsSent = embedsSent + authorData.getEmbedsSent();
    }
}

Emoji Reaction count

Comparator

public static class EmojiCountComparator implements Comparator<Emoji> {
    @Override
    public int compare(Emoji o1, Emoji o2) {
        final int compare = o2.count.compareTo(o1.count);

        if (compare == 0) {
            return 1;
        } else {
            return compare;
        }
    }
}

Analysis

private void calculateMostCommonReaction(List<AuthorData> authorDataList) {
    Map<Emoji, Integer> emojiCount = new HashMap<>();
    authorDataList.forEach(authorData -> {
        final Map<Emoji, Integer> emojisReceived = authorData.getEmojisReceived();
        emojisReceived.forEach((key, value) -> {
            if (emojiCount.containsKey(key)) {
                emojiCount.put(key, emojiCount.get(key) + value);
            } else {
                emojiCount.put(key, value);
            }
        });
    });

    // write emoji count
    for (Map.Entry<Emoji, Integer> emoji : emojiCount.entrySet()) {
        emoji.getKey().setCount(emoji.getValue());
    }
    mostCommonReaction.putAll(emojiCount);
}

Messages sent

Comparator

public static class AuthorDataMessagesCountComparator implements Comparator<AuthorData> {
    @Override
    public int compare(AuthorData o1, AuthorData o2) {
        final Long messagesSent1 = o1.getMessagesSent();
        final Long messagesSent2 = o2.getMessagesSent();
        final int compare = messagesSent2.compareTo(messagesSent1);

        if (compare == 0) {
            return 1;
        } else {
            return compare;
        }
    }
}

Analysis

private void calculateMessageRanking(List<AuthorData> authorDataList) {
    mostMessages = new TreeMap<>(new AuthorData.AuthorDataMessagesCountComparator());
    authorDataList.forEach(authorData -> mostMessages.put(authorData, authorData.getMessagesSent()));
}

private void countMessages(List<AuthorData> authorDataList) {
    for (AuthorData authorData : authorDataList) {
        messagesSent = messagesSent + authorData.getMessagesSent();
    }
}

Times Mentioned

Comparator

public static class AuthorDataMentionsCountComparator implements Comparator<AuthorData> {
    @Override
    public int compare(AuthorData o1, AuthorData o2) {
        final Long timesMentioned1 = o1.getTimesMentioned();
        final Long timesMentioned2 = o2.getTimesMentioned();
        final int compare = timesMentioned2.compareTo(timesMentioned1);

        if (compare == 0) {
            return 1;
        } else {
            return compare;
        }
    }
}

Analysis

private void calculateMentionRanking(List<AuthorData> authorDataList) {
    timesMentioned = new TreeMap<>(new AuthorData.AuthorDataMentionsCountComparator());
    authorDataList.forEach(authorData -> timesMentioned.put(authorData, authorData.getTimesMentioned()));
}

private void countMentions(List<AuthorData> authorDataList) {
    for (AuthorData authorData : authorDataList) {
        countMentions = countMentions + authorData.getTimesMentioned();
    }
}

UML

Zoom in to read

Download UML Image