HW 8

Readme.md

@@ -1,53 +1,53 @@
-# Задание №8
+# Task #8
 
-В этом задании вам нужно написать `case-study` о том как вы применили знания, полученные на курсе, к своим проектам.
+In this task you need to write a `case-study` about how you applied the knowledge gained in the course to your projects.
 
 ## To start
 
-Для начала напишите немного о своём проекте.
+To begin, write a little about your project.
 
-- что за проект
-- как долго уже разрабатывается
-- как дела с перформансом
-- есть ли мониторинг
-- можете ли вы навскидку предположить где в проекте есть что оптимизировать
-- какова ваша роль в проекте, как давно работаете, чем занимаетесь
+- what kind of project is it
+- how long has it been in development
+- how is performance going
+- is there monitoring
+- can you roughly guess where in the project there is something to optimize
+- what is your role in the project, how long have you been working, what do you do
 
-Сделайте `PR` в этот репозиторий, и дорабатывайте его по ходу курса.
+Make a `PR` to this repository, and refine it as the course progresses.
 
 ## Hints
 
-Форма `case-study` - свободная.
+The `case-study` format is free-form.
 
-Можно написать в форме интересной технической статьи на Хабр. Потом можно будет и опубликовать.
+You can write it in the form of an interesting technical article for Habr. Then you can publish it.
 
-Можно взять за основу форму `case-study` из первого задания.
+You can use the `case-study` form from the first task as a basis.
 
 ### MVP is OK
 
-Оптимизация не обязана быть доведена до прода.
+The optimization doesn't have to be brought to production.
 
-Например, вы рассмотрели какую-нибудь подсистему с `fullstack` точки зрения и придумали как её оптимизировать, сделали `MVP`, получили первые результаты.
+For example, you considered some subsystem from a `fullstack` point of view and figured out how to optimize it, made an `MVP`, got first results.
 
-В таком случаем интересно рассказать об этом.
+In that case, it's interesting to talk about it.
 
 
-### О чём интересно рассказать
+### What's interesting to talk about
 
-- расскажите об актуальной проблеме;
-- расскажите, какой метрикой характеризуется ваша проблема;
-- если вы работали в итерационном процессе оптимизации, расскажите как вы построили фидбек-луп;
-- если пользовались профайлерами - опишите находки, которые сделали с их помощью;
-- расскажите, как защитили достигнутый прогресс от деградации;
-- прикиньте, сколько денег сэкономила ваша оптимизация: сократили потребление памяти и сэкономили денег на серверах / ускорили ответ сервера и уменьшили bounce-rate / ускорили прогон тестов и улучшили рабочий feedback-loop для всех участников команды...; если сделали что-то полезное, но сложно понять, как это оценить в деньгах, пишите в `Slack`, обсудим;
-- если вы сделали много оптимизаций, расскажите о всех! чем больше - тем лучше! если какие-то из них менее интересны, упомяните о них обзорно;
+- tell about the actual problem;
+- tell what metric characterizes your problem;
+- if you worked in an iterative optimization process, tell how you built the feedback-loop;
+- if you used profilers - describe the findings you made with their help;
+- tell how you protected the achieved progress from degradation;
+- estimate how much money your optimization saved: reduced memory consumption and saved money on servers / sped up server response and reduced bounce-rate / sped up test runs and improved the working feedback-loop for all team members...; if you did something useful but it's hard to understand how to evaluate it in money, write in `Slack`, we'll discuss;
+- if you made many optimizations, tell about all of them! the more the better! if some of them are less interesting, mention them in overview;
 
-### Если ничего не приходит в голову
+### If nothing comes to mind
 
-Всегда можно оптимизировать тесты вашего проекта с помощью `test-prof`! (если конечно они уже не доведены до идеала)
+You can always optimize your project's tests using `test-prof`! (unless of course they're already perfected)
 
-Всегда можно сделать аудит проекта с помощью `sitespeed.io`, `webpagetest`, `pagespeed insights`, `lighthouse` и применить предложенные советы.
+You can always audit your project using `sitespeed.io`, `webpagetest`, `pagespeed insights`, `lighthouse` and apply the suggested tips.
 
-## Как сдать задание
+## How to submit the task
 
-Сделайте `PR` в этот репозиторий с вашим `case-study`.
+Make a `PR` to this repository with your `case-study`.

case-study.md

@@ -0,0 +1,60 @@
+## About the project
+
+I work at Jetrockets company.
+
+We're writing a project for an American law firm, implementing a unified application that meets their needs, previously they had to use various software for different tasks, now with our application everything is in one place.
+
+The project has been in development since the end of February 2020.
+
+I joined the project and the company in general at the end of January 2021, in the company I am a back-end developer in ruby.
+
+The application is an API that communicates with a separate frontend written entirely in Svelte.
+
+Performance in the project wasn't really addressed since the project is intended for a corporate client,
+No monitoring systems are used, except for sending errors to rollbar
+
+## Stack
+0. Rails 6
+1. Grape for writing API
+2. Dry-rb for validations and initializations
+3. PostgreSQL
+4. Rspec
+
+## Case 1
+
+Started optimizing the project from the most painful point, which is report generation, it could be viewed in the web version and exported to different formats.
+
+The problem was that this was happening quite slowly, especially on my computer, which had a regular hdd,
+sometimes I had to go to the server where ssd is used to check how the report is generated.
+
+We had been planning this for a long time and when the opportunity arose we moved the work to sidekiq, I don't know how, but it started working tens of times faster, I still couldn't figure out why there was such a colossal difference in time between synchronous and asynchronous execution, perhaps keeping the connection open during export was slowing something down.
+
+## Case 1.1
+
+I decided to speed up the export even more and started with the code, profiled it with all the tools studied and there were no special problems with the code, it was written quite well, I found a couple of pain points that didn't play a special role in the speed of forming the report for export,
+
+then I decided to switch to the database, started with pgHero, it showed me slow queries, but their cause was not clear, switched to NewRelic, thankfully now it's free for one user, collected a little analytics and noticed that a composite field search is often used in the query, I added an index and managed to achieve about 30% speedup
+also I noticed that a junction table is often used, did a reload and magically this helped achieve a twofold speedup of export.
+
+The problem that remains relevant is that during export we load all records into RAM since a complex sort is used in the selection, and such sorting cannot be done in batches.
+
+
+## Case 2
+
+Got into tests, tests were running for 10 minutes in a single thread, and in parallel_tests ~3 minutes.
+
+Ran test-prof for repeated let calls and replaced them with let_it_be, with minimal effort managed to speed up the export by two times to 1.4 minutes (+1905 tests), there's still a lot that can be optimized but stopped at this for now.
+
+### Summary
+
+In the end, managed to make life easier for clients by speeding up their export, and for developers by speeding up their tests.
+
+Unfortunately the project is at the completion stage, we're literally delivering it next week and many things won't be possible to apply.
+
+But at least colleagues appreciated the speedup of test runs and test-prof will definitely become part of the project.
+
+### Problems during optimization
+
+Since Grape is used as the Api, NewRelic didn't understand very well what was happening in the resources, and on the graphs Grape execution could take 80% of the time and it's unclear what's inside.
+
+let_it_be should also be applied carefully since it creates unstable tests, sometimes they pass, sometimes they don't, it apparently has some problems with associations that are created inside factories.