Big Data Analysis Using Distributed Approach on Weather Forecasting Data

Amit Palve¹ , Ajit Patil² , Amol Potgantwar³

¹ Computer Engineering Department, SITRC (Pune University), Nashik, India.
² Computer Engineering Department, SITRC (Pune University), Nashik, India.
³ Computer Engineering Department, SITRC (Pune University), Nashik, India.

Correspondence should be addressed to: ajit.patil1091@gmail.com.

Section:Research Paper, Product Type: Journal
Vol.5 , Issue.3 , pp.39-43, Jun-2017

Online published on Jun 30, 2017

Copyright © Amit Palve, Ajit Patil, Amol Potgantwar . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
 

View this paper at   Google Scholar | DPI Digital Library

XML View     PDF Download

How to Cite this Paper

1287 Views    461 Downloads    422 Downloads

Abstract :
Data than it seems at first, and extracting the useful information in an efficient manner leads a system toward a major computational challenges, such as to analyze, aggregate, and store, where data are remotely collected. Keeping in view the above mentioned factors, there is a need for designing a system architecture that welcomes both real-time, as well as offline data processing. Big data is data whose characteristics force us to look beyond the traditional methods that are prevalent at the time. Online news, micro-blogs, search queries are just a few examples of these continuous streams of user activities. Evolving data streams methods are becoming a low-cost, green methodology for real time online prediction and analysis. Heterogeneity, scale, timeliness, complexity, and privacy problems with Big Data impede progress at all phases of the pipeline that can create value from data. The problems start right away during data acquisition, when the data tsunami requires us to make decisions, currently in an ad hoc manner, about what data to keep and what to discard, and how to store what we keep reliably with the right metadata.

Key-Words / Index Term :
Big data, ETL, Data processing unit, RSDU, DADU, real-time

References :
[1] Muhammad Mazhar Ullah Rathore, Anand Paul , "Real-Time Big Data Analytical Architecture for Remote Sensing Application�, IEEE Journal of Selected Topics In Applied Earth Observations And Remote Sensing, Vol.8, No.10, pp.12-19, 2015.
[2] R. A. Dunge, “A Survey on Big Data in Real�, IJRITCC, Vol No. 2, issue.4, pp.1-5, 2015.
[3] D. Agrawal, S. Das, A.E. Abbadi, "Big Data and cloud computing: Current state and future opportunities,� in Proc. Int. Conf. Extending Database Technol. (EDBT),India, pp.530-533, 2011.
[4] J. Cohen, B. Dolan, M. Dunlap, J. M. Hellerstein, C. Welton, "Mad skills: New analysis practices for Big Data�, PVLDB, vol. 2, no. 2, pp. 1481-1492, 2009.
[5] A. Cuzzocrea, D. Sacc, J.D. Ullman, "Big Data: A research agenda�, in Proc. Int. Database Eng. Appl. Symp. (IDEAS13), Barcelona, Spain, pp.09-11, 2013.
[6] R.A. Schowengerdt, "Remote Sensing: Models and Methods for Image Processing�, Academic Press, USA, pp.1-6, 1997.
[7] D. A. Landgrebe, "Signal Theory Methods in Multispectral Remote Sensing�, Hoboken Wiley, USA, pp.6-11, 2003.
[8] C.-I. Chang, "Hyperspectral Imaging: Techniques for Spectral Detection and Classification�, Norwell, USA, pp.90-97, 2003.
[9] J. A. Richards, X. Jia, "Remote Sensing Digital Image Analysis: An Introduction�, Springer, USA, pp.34-41, 2006.
[10] J. Shi, J. Wu, A. Paul, L. Jiao, M. Gong, "Change detection in synthetic aperture radar image based on fuzzy active contour models and genetic algorithms�, Math. Prob. Eng., Vol. 2014, Issue.4, pp.1-15, 2014.
[11] A. Paul, J. Wu, J.-F. Yang, J. Jeong, "Gradient-based edge detection for motion estimation in H.264/AVC�, IET Image Process., vol. 5, no. 4, pp. 323-327, 2011.
[12] A. Paul, K. Bharanitharan, and J.-F.Wang, "Region similarity based edge detection for motion estimation in H.264/AVC�, IEICE Electron. Express, vol. 7, no. 2, pp. 47-52, 2010.
[13] A.-C. Tsai, A. Paul, J.-C. Wang, and J.-F. Wang, "Intensity gradient technique for efficient intra prediction in H.264/AVC�, IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 5, pp. 694-698, 2008.
[14] A. Plaza et al., "Recent advances in techniques for hyperspectral image processing�, Remote Sens. Environ., vol. 113, Issue.1, pp. 110-122, 2009.
[15] S. Kalluri, Z. Zhang, J. JaJa, S. Liang, J. Townshend, "Characteriz-ing land surface anisotropy from AVHRR data at a global scale using high performance computing�, Int. J. Remote Sens., vol. 22, Issue.11, pp. 2171-2191, 2001.