“Have your experiences with reflections always been positive?”
Although the dictionary definition of ‘reminiscence’ implies simply reflecting on past events, the reality is sometimes recalling those events can prove challenging. As the sprint retrospective rolls around every three weeks, I often encounter a Slack message prompting me to ‘Please fill out the memoir,’ which triggers the realization that there were important points I intended to address from the previous session. There have been several instances where I struggled to recollect specific details solely through contemplation.
Our Team Legend Trio seized the opportunity presented by the in-house hackathon, 9oormthon RAID Winter, to address the challenges associated with retrospection by leveraging a range of technologies.
Allow me to narrate the journey of how minor concerns surrounding retrospectives culminated in the development of a Slack bot named ‘Records Collection goormee’.
written by Roah, Wynter, Dominik
edit by snow, somma
Team Legend Trio comprised of three members
Roa_ Good day, I’m Roah, currently serving as an infrastructure engineer within the goorm SRE Squad. My involvement in the inaugural in-house hackathon, 9oormthon Raid, last summer was truly fulfilling, prompting my swift decision to participate as a developer in the subsequent event. My motivation for joining the 9oormthon Raid stemmed from a desire to leverage generative AI effectively, aiming to develop a product tailored to alleviate the typical challenges encountered by goormee.
Wynter_ Greetings, I’m Wynter. In 2021, I began my journey with IDE SQD as an ICT intern, and now, I’m in my second year as a full-stack developer. Although my name may differ from Espa Winter, my passion for technology remains unwavering. Participating in a hackathon has long been a goal of mine, and I enthusiastically seized the opportunity when it presented itself. During the 9oormthon Raid, I ventured into various domains, dabbling in Slackbot backend development, Node.js, and Python programming.
Dominik_ Hello, I’m Dominik, a participant in the 7th 9oormthon held in Jeju. I’ve been interning happily at goorm for the past three months since encountering the company for the first time during the event. Besides my involvement in the 9oormthon in Jeju, I’m delighted to have achieved commendable results in the 9oormthon Raid as well. During this 9oormthon Raid, I took on responsibilities in planning and Python development to some extent.
goorm boasts a robust retrospective culture, where team members come together to collectively analyze their roles, identify strengths, and pinpoint areas for improvement during the sprint period. This collaborative reflection not only enhances productivity but also fosters a sense of unity among goorm employees, facilitating their growth and evolution as a cohesive team.
Retrospective review
But retrospection is not easy. We are all going through many difficulties. One of the problems our team analyzed in the retrospective topic selection process was bias over time. As I relied on memories from the time rather than clear records to recall topics for recollection, I tended to focus more on and skim only relatively recent events.
Retrospection poses its challenges, especially considering the multitude of difficulties we encounter daily. During our team’s retrospective topic selection process, we identified bias over time as a significant issue. Relying solely on memory to recall past events often leads to a tendency to prioritize and skim through more recent occurrences, potentially overlooking valuable insights from earlier periods.
Lastly, we recognized the necessity for a tool that provides administrators, rather than members, with a comprehensive overview of the organization’s tasks, akin to a dashboard. Previously, we relied on individual members to summarize their work during each sprint, which proved challenging amidst the hustle and bustle of daily tasks, making it difficult to compile meaningful reports. Consequently, we sought to collaboratively address these requirements.
To address these diverse challenges, the team reached a consensus on the necessity of implementing a Slackbot to enhance the convenience of retrospectives and cultivate a retrospective culture. It was apparent that mitigating time-dependent bias could be achieved by leveraging generative AI to analyze Slack conversations spanning the sprint duration, thereby autonomously generating retrospective documents and organizing summaries.
Developing a retrospective manager to facilitate retrospectives.
The service developed to address these concerns is called ‘Record Collection goormee’. It serves as a retrospective manager aimed at facilitating retrospectives. By analyzing prevalent keywords and conversation history derived from records within the Slack channel throughout the sprint period, ‘Record Collection goormee’ suggests three retrospective topics. Its primary function is to assist in gathering records, thereby enhancing the productivity of retrospectives.
Usage is straightforward. Simply type “/collect records” in your Slack channel and await the AI-generated retrospective topics shortly thereafter.
▲ Witness goormee in action, swiftly generating insights after collecting records.
This is how it works
Record Collection goormee operates with two servers, Express.js and Flask.
The Express server initially receives user /commandslash commands and requests data processing from the Flask server, including channel information. Simultaneously, to prevent Slack’s message timeout error, a response value indicating a brief processing period is sent to the user.
The Flask server utilizes the Slack API to obtain comprehensive message data from the ongoing sprint, based on the provided channel information. Nouns are extracted from the received messages using a natural language processing function, following which the most frequent words and sentences containing those words are identified. Subsequently, the extracted data is stored in the MongoDB database, alongside Slack channel and sprint details.
Having received confirmation of data processing completion, the Express.js server proceeds to query the database for the most frequent words and sentences. Utilizing the gathered insights, formulate a GPT prompt to solicit recommendations for retrospective topics.
Lastly, transmit the outcome of the GPT request back to the Slack channel, thus finalizing all operations.
What aspects did you prioritize or focus on?
- Data Preprocessing
Conversations within Slack serve as valuable indicators of the tasks prioritized by an organization throughout the sprint. For instance, frequent mentions of keyword A during this period may suggest that topics related to A are pertinent for inclusion in the retrospective agenda.
In order to achieve this, our approach involved extracting a comprehensive set of sentences not only from main threads within the Slack channel but also from replies within heavily commented threads. Regrettably, due to certain limitations, we were unable to access every thread and its replies. This decision was necessitated by the need to develop a Minimum Viable Product (MVP) within the constraints of a brief hackathon timeframe.
The extracted data underwent part-of-speech tagging using KoNLPy’s Okt, with a subsequent focus on isolating only the nouns. Given the inherent complexity of sentence-type data, the intent was to utilize it primarily as a search key, facilitating the extraction of essential information.
Subsequently, the extracted nouns were arranged in descending order of frequency, and sentences containing these nouns were sequentially searched, starting from the most frequently occurring ones. For instance, if the noun ‘credit’ featured prominently within a specific channel, all corresponding sentences were retained for further processing.
def user_freq_extract(messages_json2):
extract_raw = json_data(messages_json2)
user_list = list(extract_raw.keys())
user_freq_dict = {}
okt = Okt()
for user in user_list:
user_freq_dict[user] = Counter()
for POS_sentence in extract_raw[user]:
words = okt.nouns(POS_sentence)
word_frequencies = Counter(words)
user_freq_dict[user] += word_frequencies
return user_freq_dict▲ Illustrative code showcasing the tagging of parts of speech and computation of word frequency utilizing the Okt module.
2. Data to GPT to Slack
The sentence data acquired through this procedure is transmitted to GPT. Within GPT, a virtual persona named ‘Retrospective Manager’ is established. Prompt engineering is conducted to extract insightful retrospective topics from the provided input sentences.
Our team encountered significant challenges, notably the inability to integrate OpenAI’s GPT API due to its limited understanding of the English language and its tendency to disregard clearly defined prompts, resulting in inappropriate behavior. Throughout this ordeal, the criticality of crafting effective GPT prompts became abundantly clear to me.
Initially, we adopted a straightforward approach by crafting a prompt requesting individuals to conduct a retrospective analysis based on conversations associated with the most prevalent keywords. However, due to the abstract nature of the prompt, the responses generated by GPT exhibited inconsistencies and included extraneous information, such as the keywords provided for analysis and redundant reiterations of conversations.
To address this issue, we iteratively refined the prompts, gradually honing them to provide clearer instructions. Ultimately, we assigned GPT the role of goorm’s retrospective manager.
exports.getSystemRoleTemplate = () => `
You are now the retrospective manager who will assist our team's retrospectives.
Your role is to analyze Slack conversations and keywords given to create retrospective topics.
`;
exports.getMessageTemplate = (keyword, sentences) =>`
... omitted
Now, based on the most frequently occurring keyword list and the list of sentences where keywords are mentioned below, please generate three retrospective topics and write them according to the output template.
Most frequent keyword list: ${keyword}
List of sentences where keywords are mentioned: ${sentences}
// Specify the output format directly
<Output Template>
Retrospective Topic: ''
Explanation of the retrospective topic: ''
You must send three outputs in this format exactly according to the template.
Do not send the most frequent keyword list and the list of sentences where keywords are mentioned again. // Request to remove unnecessary data
`;
▲ Template example code
By providing GPT with a detailed role that elucidated the context, and by explicitly specifying the template for GPT’s output, we successfully mitigated the issue of inconsistent responses.
Additionally, by explicitly defining the output template in this manner, I was able to seamlessly transmit the response results to Slack without the need for significant format adjustments.
Our hackathon journey continues.
Throughout the week-long in-house testing phase, colleagues who utilized Record Collection goormee provided us with numerous evaluations. In addition to positive feedback, their diverse perspectives, including suggestions for improvement and areas of concern, proved invaluable in guiding our future course of action.
“I appreciated the recommendations for relevant topics based on our conversations during retrospectives. It helped alleviate the challenge of determining discussion points.”
“I often found myself missing important details during retrospectives due to the volume of content exchanged within a single sprint. I believe this tool will prove beneficial in addressing this issue.”
“While I value the convenience of extracting Slack conversation content, I have concerns about the security implications of sharing such data with a public AI.”
“I would appreciate it if the tool could suggest more retrospective topics, as I believe the current three recommendations fall short of addressing all relevant areas.”
Although the in-house hackathon has concluded, our journey of innovation persists. We remain committed to enhancing Record Collection goormee based on valuable feedback and further iterations of our Minimum Viable Product(MVP). We invite you to stay tuned for the ongoing developments and the eventual establishment of a company-wide culture centered around the use of Record Collection goormee for retrospectives. ☁️
ROA_ Participating in hackathons often ignites my motivation to explore a broader range of endeavors in the future. When the idea of a retrospective bot surfaced during our initial brainstorming session, I initially perceived the core functionalities—extracting conversation content and analyzing keywords—as technically complex tasks that would require more than two days to complete. Admittedly, the prospect of having to utilize both Python for keyword analysis and Node.js for Slack integration, as well as interfacing with GPT, presented a significant challenge.
Contrary to my initial concerns, I found myself swiftly setting up a server using Python and Flask, technologies with which I had limited experience. Witnessing my team members develop a keyword analysis function, I realized the beauty of embracing challenges and adopting a challenger mindset. I felt ashamed for doubting my abilities before even beginning the task. This hackathon served as a pivotal starting point, reaffirming the importance of approaching challenges with a fearless mindset, free from unnecessary worries.
Wynter_ Participating in the hackathon posed several concerns for me, ranging from physical endurance to skill level and unfamiliar environments. Interestingly, the aspects I fretted about indeed proved to be formidable challenges. However, amidst this journey of facing challenges head-on, I rediscovered my passion for development. Much like my initial foray into development, grappling with a new language left me feeling somewhat lost. Yet, witnessing the gratifying results of my efforts functioning seamlessly reignited my enthusiasm. Ultimately, the most significant achievement for me was regaining the motivation to pursue my development career with renewed vigor.
Dominic_ Reflecting on my journey in overcoming the challenges of 9oormthon, I ponder whether I have attained mastery. Back in 2022, when I participated in the 7th 9oormthon in Jeju, little did I anticipate interning at 9oormthon or engaging in the in-house hackathon ‘9oormthon’ once again. This experience underscores the unpredictable nature of life’s twists and turns.
Initially embarking as a planner with ambitious goals, I grappled with concerns about meeting deadlines. However, the unwavering support of exceptional colleagues like Roa and Wynter proved invaluable in realizing our Minimum Viable Product (MVP). I extend my heartfelt gratitude to these individuals for their pivotal role in executing our ambitious vision.
