Comprehensive Design Flow for Effective Data Processing and Visualization

Authors

  • Samudyata A Department of Electronics and Telecommunication Engineering, R V College of Engineering, Bangalore, India
  • P Nagaraju Department of Electronics and Telecommunication Engineering, R V College of Engineering, Bangalore, India

DOI:

https://doi.org/10.24203/ijcit.v12i2.347

Keywords:

Talend, Power BI, Data Analytics, Exploration

Abstract

Data is pervasive in the modern digital world. Data is continually being created, gathered, and analysed, from social media interactions to financial transactions. Businesses face both opportunities and challenges due to the data boom. On one hand, data may be a useful resource for companies looking to differentiate themselves from the competition, spur innovation, and enhance decision-making. However, the sheer amount and complexity of data can be daunting, making it challenging to glean insightful information and take timely action on it. Organisations need a strong and well-designed data processing design flow that can allow effective data exploration and creative visualization in order to meet this challenge. This work focuses on the complete data processing lineage and integrating that with various tools such as Talend and Power BI for data analytics to deliver diverse visualizations and insights which can cater businesses to stay ahead of the curve, providing a foundation for success in the ever-evolving digital landscape.

References

H. Ashok, S. Ayyasamy, A. Ashok and V. Arunachalam, "E-business Analytics through ETL and Self-Service Business Intelligence Tool," 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2020, pp. 747-751, doi: 10.1109/ICIRCA48905.2020.9183336.

W. Jijun, C. Yongqiu and C. Li, "Research on Multi-layer Power Enterprise Data Management Architecture Based on Big Data," 2022 IEEE 8th International Conference on Computer and Communications (ICCC), Chengdu, China, 2022, pp. 1194-1198,

Y. Mao, S. Huang, S. Cui, H. Wang, J. Zhang and W. Ding, "Multidimensional data distribution monitoring based on OLAP," 2020 2nd International Conference on Information Technology and Computer Application (ITCA), Guangzhou, China, 2020, pp. 298-302, doi: 10.1109/ITCA52113.2020.00070.

S. Yi and H. Ye, "A Practical Workflow in Cleaning Master Data," 2020 IEEE International Conference on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom), Exeter, United Kingdom, 2020, pp. 1311-1314

A. Kaminskyi and M. Nehrey, "Information Technology Model for Customer Relationship Management of Nonbank Lenders: Coupling Profitability and Risk," 2021 11th International Conference on Advanced Computer Information Technologies (ACIT), Deggendorf, Germany, 2021,pp. 234-237

J. Sreemathy, S. Priyadharshini, K. Radha, K. Sangeerna and G. Nivetha, "Data Validation in ETL Using TALEND," 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS), Coimbatore, India, 2019, pp. 1183-1186, doi: 10.1109/ICACCS.2019.8728420.

M. Riasetiawan, F. Anggara, A. Ashari, S. Winardi and B. N. Prastowo, "Data Model and Analysis for Big Data Mapping and Management in the Energy Data Platform," 2021 International Conference on Data Science, Artificial Intelligence, and Business Analytics (DATABIA), Medan, Indonesia, 2021, pp. 1-6,

J. Sreemathy, I. Joseph V., S. Nisha, C. Prabha I. and G. Priya R.M., "Data Integration in ETL Using TALEND," 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 2020, pp. 1444-1448, doi: 10.1109/ICACCS48705.2020.9074186.

J. Sreemathy, R. Brindha, M. Selva Nagalakshmi, N. Suvekha, N. Karthick Ragul and M. Praveennandha, "Overview of ETL Tools and Talend-Data Integration," 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 2021, pp. 1650-1654, doi: 10.1109/ICACCS51430.2021.9441984.

P. S. Diouf, A. Boly and S. Ndiaye, "Variety of data in the ETL processes in the cloud migration and validation : State of the art," 2018 IEEE International Conference on Innovative Research and Development (ICIRD), Bangkok, 2018, pp. 1-5.

M. Balakrishnan, N. M, K. Ramya and S. K, "Cloud Computing based Data Validation and Migration in ETL using Talend," 2022 6th International Conference on Electronics, Communication and Aerospace Technology, Coimbatore, India, 2022, pp. 1349-1355, doi: 10.1109/ICECA55336.2022.10009262.

R. Akbar, Wahyudi, J. Rahmadoni and H. A. Lubis, "The Implementation of Business Intelligence on Visualisation of Transaction Data Analysis using Dashboard System Case Study: XYZ Convenience Store," 2022 International Symposium on Information Technology and Digital Innovation (ISITDI), Padang, Indonesia, 2022, pp. 42-49, doi: 10.1109/ISITDI55734.2022.9944494.

M. Zhang, J. Zhou, Q. Wang and X. Zhang, "Data Quality Management for Big Data: A Review," in IEEE Access, vol. 7, pp. 70102-70114, 2019

Downloads

Published

2023-06-30

How to Cite

A, S., & P Nagaraju. (2023). Comprehensive Design Flow for Effective Data Processing and Visualization. International Journal of Computer and Information Technology(2279-0764), 12(2). https://doi.org/10.24203/ijcit.v12i2.347

Issue

Vol. 12 No. 2 (2023): June 2023

Section

Articles

License

Copyright (c) 2023 Samudyata A, P Nagaraju

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

The articles published in International Journal of Computer and Information Technology (IJCIT) is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.