Case Studies

Real Estate Sales Recommendation Engine

Real Estate Sales Recommendation Engine
The real estate sales platform is a service for fast and convenient selling real property. A seller visits the site and answers certain questions to get a piece of advice. The consumer adds information about the address of the building and receives a recommendation on an approximate house price and the best sales method, real estate agent, cash offer or real estate company
Challenge 1/4:

Ingest about 1TB of data to DB and run this ingestion continuously once per month for new data

Solution:

Built ETL process on AWS DataPipeline using Bash and NodeJS that ingest all data and create the necessary indexes/slices/views for less than 12 hours

Challenge 2/4:

Run geospatial queries on large tables > 200MLN records for less than 300ms

Solution:

Decided to use AWS Aurora as a main operational database and PostGIS extension to run geospatial queries and added required indexes and optimizations to make the execution as fast as possible

Challenge 3/4:

Integrate with more than 20 external API and 10 microservices through REST

Solution:

Implemented declarative utility client on top of base HTTP client with built-in backoff, retries, and circuit breaker. Added common library and whole architecture for all microservices in a system

Challenge 4/4:

Implement intellectual recommendation engine

Solution:

Developed the recommendation engine, that takes a lot of answers and other inputs from external sources and makes a decision about the best selling option

<Previous>Next

Languages

JavaJava
JavaScriptJavaScript
Bash - Unix shellBash - Unix shell

Databases

PostgreSQLPostgreSQL
MySQKMySQK
MongoDBMongoDB
RedisRedis
AWS RedshiftAWS Redshift
BigQueryBigQuery

Tools

SpringSpring
Node.jsNode.js
AWS S3AWS S3
LambdaLambda
AWS SQSAWS SQS
DockerDocker
JenkinsJenkins
SwaggerSwagger

Ad Management Platform

Ad Management Platform
The Ad Management Platform is developed for effective controlling ads on Google Ads and Bing Ads. The system is able to create millions of ad groups and keywords and manages every advertisement based on internal rules. The largest number of ads was created during Black Friday and Cyber Monday. It was about 100 million keywords.
Challenge 1/4:

Find the best database that will support doing about 100K updates per second and can scale linearly

Solution:

We've investigated a lot about different databases and made a decision to use the Apache Cassandra cluster and fine-tune the system to meet the client’s requirement

Challenge 2/4:

Implement the platform that will be able to manage any amount of ads at the same time

Solution:

The client received a scalable system for creating and controlling any amount of advertising. We had about 40 servers in production. 12 of them were Apache Cassandra servers and 28 were the application servers. We used RabbitMQ for distributing the messages across the system

Challenge 3/4:

Reduce the infrastructure cost, implement on-demand scaling

Solution:

The solution was to implement ETL on-the-fly data streaming in BigQuery using the streaming API, in order to decrease reports generation and data warehouse cost. Implemented the system that allows to deploy or stop some servers depend on different metrics - CPU load, RAM usage, etc)

Challenge 4/4:

Develop the real-time model that process every single click and make a decision about the ad - change a price or stop it

Solution:

We've implemented a model of tracking and price changes in real time on top of DynamoDB.It processed every click and made a decision on advertising on the fly

<Previous>Next

Languages

JavaJava
JavaScriptJavaScript
ClojureClojure

Databases

PostgreSQLPostgreSQL
CassandraCassandra
BigQueryBigQuery
MongoDBMongoDB
RedisRedis
AWS RedshiftAWS Redshift

Tools

SpringSpring
Node.jsNode.js
AWS S3AWS S3
LambdaLambda
DockerDocker
JenkinsJenkins
AnsibleAnsible
BootstrapBootstrap

Reporting Module for SaaS

Reporting Module for SaaS
The business intelligence module for an already existing SaaS beauty salon solution. The application is fully responsible for the reports system of each salon (customers, discounts, employees, tips, etc.) and the salon network as well
Challenge 1/4:

Find a way to ingest data from several MySQL instances to another database

Solution:

Implemented a service that listened for MySQL changelogs and put all changes to Apache Kafka and another service that read data from Kafka and put to any database

Challenge 2/4:

Find a solution that fits great for the large and small amount of data

Solution:

Usage of a hybrid solution for reports, Impala for large amounts of data, as well as for medium and MemSQL for small reports

Challenge 3/4:

Speed up the creation of reports for large data volumes of 100GB+

Solution:

Decided to generate a fact table from the main, frequently used tables to avoid unnecessary joins in order to speed up the report's creation. Another option for speeding up reports was to move some often changed data to in-memory storage

Challenge 4/4:

Implement more than 100 custom reports

Solution:

Implemented reports in pure SQL, wrote and maintained code for fairly large reports. Used plain SQL allowed us to change source databases with only small effort

<Previous>Next

Languages

JavaScriptJavaScript
SQLSQL
Bash - Unix shellBash - Unix shell

Databases

MySQKMySQK
MemSQLMemSQL
ImpalaImpala
RedisRedis
HadoopHadoop

Tools

Node.jsNode.js
AWS S3AWS S3
LambdaLambda
DockerDocker
JenkinsJenkins
AnsibleAnsible
RancherRancher
SwaggerSwagger

Business Intelligence Solution

Business Intelligence Solution
The SaaS allows putting various data sources (including SQL and NoSQL DBs, CSV, XML, XLS, JSON, etc.) into the system and builds reports based on this data. The user could filter, aggregate, sort and download the processed data
Challenge 1/4:

Find the best approach to upload and index large files (50GB+) through the web

Solution:

We’ve developed a chunk uploader for efficiently work in spite of a slow Internet. The user was able to close a browser/tab and continue uploading whenever he wants

Challenge 2/4:

Find a solution to handle large and small files efficiently

Solution:

Developed a hybrid solution to data ingestion. Large files were put into BigQuery, while medium and small files were put into ElasticSearch. This solution allowed us to process very large files and run analytics queries for less than 2 seconds on large datasets

Challenge 3/4:

Determine file format and the approximate progress of the file ingestion

Solution:

The solution was to implement a special logic, based on the heuristic algorithm to successfully overcome the challenge and show the right progress to the end user

Challenge 4/4:

Development of a rather reach and fast interface

Solution:

Found the canvas table, forked and customized it for our needs. It’s a similar way as Google Sheets works, with lazy loading and rendering only those data that the user sees

<Previous>Next

Languages

JavaJava
JavaScriptJavaScript
SQLSQL

Databases

BigQueryBigQuery
ElasticsearchElasticsearch
PostgreSQLPostgreSQL

Tools

KibanaKibana
DockerDocker
Google Cloud PlatformGoogle Cloud Platform
RancherRancher
GitLabGitLab
AngularAngular

Mobile Application for Mountain Bikers

Mobile Application for Mountain Bikers
The system is designed for management within teams and cyclists training sessions. It allows users to plan seasons, communicate with each other, organize meetings and track the location of team members in real time
Challenge 1/3:

Build an architecture for the mobile app backend

Solution:

Developed backend on NodeJS using SaaS solution for simplifying development and reducing the initial price

Challenge 2/3:

Develop one mobile app that runs on IOS and Android

Solution:

Used ReactNative for cross-platform app development

Challenge 3/3:

The project was partially implemented. So, we had to figure out and refactor the code

Solution:

Changed the general mobile app architecture added fundamental best practices. We were fully involved in the project, participated in making technical decisions and recommended various features that were successfully implemented

<Previous>Next

Languages

JavaScriptJavaScript
SQLSQL
HTML5HTML5

Databases

PostgreSQLPostgreSQL

Tools

React NativeReact Native
Node.jsNode.js
AWS S3AWS S3
LambdaLambda
RancherRancher
ReduxRedux

Mobile Application for Dreamers

Mobile Application for Dreamers
The app is available in the Apple Store and Google Play. It’s an Instagram like social network for sharing dreams. The system allows the user to create a dream, share it with friends, follow other users and track progress on the achieving specific dream. The app is integrated with social networks
Challenge 1/3:

Working with a lot of legacy code, without comments and structuring

Solution:

Successfully completed refactoring in a short time, replaced all legacy code and technologies with newest, fast and reliable ones. It simplified further development a lot and each new feature was delivered much faster than the client expected

Challenge 2/3:

Implement a complete redesign of the application after successfully finishing the first phase

Solution:

After the successful release, we’ve made a new development iteration, that was aimed to do a complete application redesign, and themes support

Challenge 3/3:

Implementation of the backend and admin panel

Solution:

Discovered a ready admin panel, that supports customization and deployed it to the AWS, instead of developing our own solution, we found it a good practice to search for a built decision instead of developing the bicycle

<Previous>Next

Languages

JavaScriptJavaScript
SQLSQL
HTML5HTML5

Databases

PostgreSQLPostgreSQL

Tools

React NativeReact Native
ReactJSReactJS
Node.jsNode.js
AWS S3AWS S3
RancherRancher
GitLabGitLab
ReduxRedux
CSS3CSS3

Online Analytical Processing Platform

Online Analytical Processing Platform
The product is an in-memory multidimensional online server, which allows analytical processing of raw data. It performs functions of a convenient BI tool. The server receives information of different types, which can be included in many data-sources (Excel files, RDBMS, CSV files, MongoDB, etc.)
Challenge 1/5:

Optimization of server interaction on UI

Solution:

Because of large amounts of data, we made caching, pagination, lazy loading and requests throttling on UI. Implemented logs management and sending all frontend logs to the server

Challenge 2/5:

Performance optimization to show a lot of data on UI

Solution:

In order to improve performance, we’ve created dashboards that could contain any number of elements (charts, widgets, etc.), it was done through virtualization techniques, so we showed only those elements that user saw at the moment

Challenge 3/5:

Develop a complex interface for building reports and dashboards

Solution:

Implemented the ability to create reports and dashboards for various devices (mobile, tablets, desktop) based on a complex multi-level event system

Challenge 4/5:

Choose and implement various UI libraries to meet the application needs

Solution:

Used AmCharts, Google Charts and D3 UI libraries for charts, and successfully integrated and customized them to meet the requirements

Challenge 5/5:

Struggle with a legacy code

Solution:

Moved the whole application from JQuery to ReactJS

<Previous>Next

Languages

TypeScriptTypeScript
JavaScriptJavaScript
SQLSQL

Databases

PostgreSQLPostgreSQL

Tools

React NativeReact Native
ReactJSReactJS
ReduxRedux
AmChartsAmCharts
AnsibleAnsible
RancherRancher

Advanced eCommerce Platform

Advanced eCommerce Platform
The system for creating online stores and their integration with other services (delivery, payment, inventory, etc.)
Challenge 1/4:

Development of the custom site builder

Solution:

It's usually a pretty complex task to implement a site that may generate other sites or be extremely customizable, however, we implemented a lot of responsive components that users can be able to use in their sites

Challenge 2/4:

Creation of a coupon system and integration with payment services

Solution:

We’ve implemented coupon functionality, so customers can easily pay for goods on site using coupons without any conflicts with payment services

Challenge 3/4:

Implementation of a complex business logic and inventory

Solution:

Inventory is a major part of any eCommerce business. We managed to make it done automatically and tried to find edge cases and users mistakes

Challenge 4/4:

Fulltext search for services and goods

Solution:

Implemented the distributed system of multiplatform search of services and goods based on ElasticSearch with a lot of optimizations

<Previous>Next

Languages

ClojureClojure
TypeScriptTypeScript
SQLSQL

Databases

PostgreSQLPostgreSQL
ElasticsearchElasticsearch

Tools

AngularAngular
TeamCityTeamCity
ReduxRedux
AmChartsAmCharts
SwaggerSwagger
GitLabGitLab