Konrad is a software engineer at Reddit where he works for the ranking platform team creating an internal ranking system enabling other teams to launch their own content pipelines. Throughout his career he found himself attracted to startups building social products. His first Go application was jukebox powered through Slack running on a Raspberry Pi.
From Service to Platform: A Ranking System in Go
Pipedream, inspired by UNIX pipelines, is a ranking system which is used to define a series of different ranking pipelines by combining existing or adding new stages that perform ranking steps by gathering data or applying information by sorting, filtering and branching out as part of the decision flow. We use it to rank social media content but its interface definition are sufficiently generic to rank any type of entities.
Go’s interface type system and focus on writing simple and clear code helped us to create an API that optimizes for readers and contributors of the code. A single-method interface defined at the edge of the system as well as every ranking step is at the core of this design.
Part of the success which elevated Pipedream from an experimental service to a central platform in our architecture was the demonstration of Go’s concurrency. Being able to fan-out requests in parallel and combine them later on fully utilizing multi-core powered machines allowed us to make low p9x latency but also high fault-tolerance a number one priority.