Мы используем файлы cookie для быстрой и удобной работы сайта. Выберите, какие файлы cookie вы разрешаете нам использовать. Подробнее в Политике конфиденциальности.
We use cookies to ensure our website works quickly and easily. Choose which cookies you allow us to use. Learn more in our Privacy Policy.
Discuss the project
Send a request and our specialists will contact you within 1 hour.
By clicking the "Send" button, you give your unambiguous consent to the processing of your personal data to the extent and for the purposes defined in the Personal Data Processing Policy.
Get development
Submit a request and gain access to a training manual from a leading company expert!
By clicking the "Send" button, you give your unambiguous consent to the processing of your personal data to the extent and for the purposes defined in the Personal Data Processing Policy.
Developing a Fast Data Warehouse
+15%
20%
the cost of data storage has decreased
for
hardware costs have been reduced
DEVELOPMENT OF CORPORATE DATA WAREHOUSING AND DATA LAKES
public sector
Customer
IT department of the executive authority, Moscow region
CHALLENGES/FEATURES
limited project timeframe (less than 2 months)
high demands on system performance
Development of a unified information space for executive authorities for the exchange of data marts and transfer to BI systems
Task
solution
Technical solution
1. Conducted an analysis of data handling business processes.
2. Designed an architecture for a high-speed data warehouse based on Apache Druid (a high-performance, scalable, open-source distributed system for real-time data analysis).
3. Developed a high-speed storage system for working with various data formats and files: documents, reports, spreadsheets.
4. Developed a frontend (client-side application) using React and integrated it with Apache Druid’s user access management system.
5. Implemented a backend (server-side solution) based on Python.
2. Designed an architecture for a high-speed data warehouse based on Apache Druid (a high-performance, scalable, open-source distributed system for real-time data analysis).
3. Developed a high-speed storage system for working with various data formats and files: documents, reports, spreadsheets.
4. Developed a frontend (client-side application) using React and integrated it with Apache Druid’s user access management system.
5. Implemented a backend (server-side solution) based on Python.
Result
Business values
Support for large-scale projects
The solution is suitable for large-scale industry projects requiring the processing of large volumes of data:
management and analysis of large volumes of data, taking into account the specifics of each special administrative body
Ease of use and reliability
intuitive interface
support for a large number of files
horizontal scalability and fault tolerance capabilities
Quick access and analysis
Key benefits of the solution:
quick access to necessary information
high speed and reliability of data storage
simultaneous work of more than 150 users
data storage costs decreased by 15%
hardware costs were reduced by 20%
costs for supporting data storage systems were reduced by 10%
Получите рекомендации
от экспертов «ДЮК Технологии»
от экспертов «ДЮК Технологии»
Apache Druid Temporary Data Storage: Why It Worked Effectively for Loading Table Files
Article by a Data Engineer at "DUC Technologies" on Project Development Technologies on Habr
Временное хранилище данных на Apache Druid: почему это эффективно сработало для загрузки табличных файлов
By clicking the "Submit" button, you expressly consent to the processing of your personal data to the extent and for the purposes defined in the Personal Data Processing Policy.
Development of software
and Big Data solutions
and Big Data solutions
Send a request and our specialists will contact you within 1 hour.
Choose a convenient method of communication
You can attach three files up to 3 MB each. Formats: doc, docx, pdf, ppt, pptx
Сообщение об успешной отправке!