Big Data with Deep Learning (Supervised Vs. Unsupervised): A Survey

International Journal of Computer Science Engineering and Information Technology Research (IJCSEITR) ISSN (P): 2249-6831; ISSN (E): 2249–7943 Vol. 10, Issue 1, Jun 2020, 19–30 © TJPRC Pvt. Ltd.

BIG DATA WITH DEEP LEARNING (SUPERVISED VS. UNSUPERVISED): A SURVEY ABEER ANWAR AL-QASSAB & HALA JAFFAR AL-LAWATI Department of Information Technology, Al-Zahra College for Women Oman ABSTRACT Due to the dramatic increment in technology development in the last few years, a huge volume of data is being produced every day for each person. This data is coming from different types of applications (like google search, online shopping history, etc.) and it is called Big Data. Big Data represents a massive stream of data which is continuously growing and changing, and it is needed to be controlled, analyzed and monetized to help the decision makers of organizations to innovate their companies. In addition to that, this analysis impacts the existing and future technology. Artificial Intelligence and Machine Learning algorithms are being adopted to provide effective automated tools and operations on big data (like data labeling, analyzing, diagnosing and so). One of the techniques that can be used by machine learning is the “neural networks”, and in this case it is called as Deep Learning. Deep learning could be unsupervised learning which is concerned with labeling and segmenting these large amounts of various unlabeled full of noise information. Or, it could be supervised learning which is concerned with recognizing the labeled data to recognize patterns in it, and then

unsupervised deep learning techniques, clarify when and how to be used, and give an overview on the technologies and applications which are adopting them.

KEYWORDS: Big Data, Deep Learning, Supervised Learning & Unsupervised Learning

A SURVEY

to be translated into valuable insights for further implementation. This paper is a review that survey the supervised and

Received: Jan 17, 2019; Accepted: Feb 07, 2020; Published: Feb 29, 2020; Paper Id.: IJCSEITRJUN20203

INTRODUCTION The remarkable evolution of communication technologies including (emails, interactive websites, social media applications, etc.) facilitated the exchange and transfer of great amounts of data (big data) in various forms. It also led to the emergence of the need to develop systems capable of storing and utilizing such data.

Information systems have maintained and processed customer’s data in a traditional way in order to provide a variety of services like saving, retrieving and calculating these data. Today; it is much beyond that, the ability of systems to provide knowledge-based services is also required.

For example, customer now-a-days looks for websites that are capable of suggesting what suits him of goods, movies, jobs and other things. This is done through efforts of studying data, analyzing it, applying algorithms and techniques that enable systems to derive the knowledge required to do so. And from here the term “Data Science” is initiated.

BIGDATA AND DATA SCIENCE Big Data is referred to a massive volume of data which are mostly structured and unstructured, it is so large data, and it is difficult to process using traditional software techniques.

www.tjprc.org

editor@tjprc.org

20

Abeer Anwar Al-Qassab & Hala Jaffar Al-Lawati

The term Big Data may be used to be a reference to the volume of data which is used by vendors. It may also refer to the technology required by the organization to handle the large amounts of data and storage facilities [21][22].

Big data is generated as a result of the rapid development of the web technologies, social media, sales, mobile devices data, and customer contact center). It reached (312 petabytes of information uploaded every day in 2017). This number is increasing, it may exceed exabytes (1,024 petabytes) of data which is consisting billions to trillions of records of millions of people. The data is typically loosely structured data that is often incomplete and inaccessible [21][23].

As mentioned before, this Big Data is needed to be analyzed and then to direct the impact of this analysis’s results to develop the business and technologies used. This is the job of data scientist. And this is called Data Science. “Data science is the process of extracting useful insights from data by using variety of tools and Machine Learning fundamentals.

Data science uses machine learning to learn and represent the features of the behavior of the input data and build modules for them without any intervention from the human, as will be explained later.

Data science uses Artificial Intelligence (AI), Machine Learning (ML) (which is a subset of AI), and Deep Learning methodologies (which are subset ML and they make computations of multi-layer neural network) in order to analyze data, as shown in figure1.

Figure 1: Fields of Data Science.

Data Science pass through number of stages starting with Business Requirements, then Data Acquisition, Data Processing, Data Exploration, Modeling, then Optimization as shown in figure 2.

An important stage among these stages is the modeling, which is particularly a Machine Learning process.

Different types of learning algorithms would be used to accomplish the labeling and modeling steps starting from building the model, then training, testing the model, until reaching the final stage which is improving the efficiency of the trained model [8][24].

Impact Factor (JCC): 9.7093

NAAS Rating: 3.76

Big Data with Deep Learning (Supervised Vs. Unsupervised): A Survey

21

Figure 2: Stages of Data Science Which Include Modeling (ML).

ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING General Definition Artificial Intelligence(AI) can be defined as “the branch of computer science that is concerned with the automation of intelligent behavior” [1]. Or it is defined as “the general field of “intelligent-seeming algorithms” of which machine learning is the leading frontier at the moment”.

So, the type of intelligence referred to by AI makes it possible for a machine to perform cognitive functions similar to those attributed to humans. It enables the machine to analyze and interpret data and then use these interpretations for problem solving [3]. It is always learning from the data, evolving as the data evolves and reacting to what it learns. In this way, the AI system is constantly improving and adjusting its behavior to accommodate change and this is what is called Machine Learning.

Machine learning is a branch of artificial intelligence in which a system improves its performance from experience and therefore the software applications become highly accurate in predicting outcomes without any need for explicit programming.

Supervised vs. Unsupervised Machine Learning and Big Data The learning process starts by reading and absorbing the training data in order to find a useful insights and patterns from this data. The next step is about building a model that predicts the current outcome. The performance of that model will then be tested on the dataset. This process will be repeated until the model maps each input to its correct output without any human intervention. The general process of learning is shown in figure 3.

The types of learning are divided into: supervised machine learning and unsupervised machine learning, and semisupervised learning, which combines supervised and unsupervised approaches.

In supervised learning, the user trains the program to generate answers based on a known and labeled data set.

Classification and regression algorithms are used for the learning tasks (like decision trees, and support vector machines, etc.).

In unsupervised machine learning, the algorithms generate answers on unknown and unlabeled data. Data scientists commonly use unsupervised techniques for discovering patterns in new data sets. Clustering algorithms, such as K-means, are used in unsupervised machine learning [5].

www.tjprc.org

editor@tjprc.org

22

Abeer Anwar Al-Qassab & Hala Jaffar Al-Lawati

Figure 3: The General Steps for the Machine Learning Process.

Areas of Influence in Machine Learning In Machine Learning, different methods are used to learn from data sets. Some of these are symbolic approaches in which the induced rules and the examples are explicitly represented like the statistical and mathematical methods, whereby a general distinction can be made between two systems. Other type of approaches are sub-symbolic systems, which are represented by Artificial Neural Networks (the Brain Models). These work on the principle of the human brain, whereby the knowledge contents are implicitly represented [5].

DEEP LEARNING FOR BIG DATA WITH DEEP ARTIFICIAL NEURAL NETWORKS Multi-Layer Artificial Neural Networks An artificial Neural Network is a Machine Learning approach based on modeling adaptation of biological neural systems.It can also be defined as a computing system made up of number of simple highly interconnected processing elements which processes information by their dynamic state response to external inputs. Information processing is carried out through connectionist approach to computation. An example of such a neural network appears in figure 4.

Impact Factor (JCC): 9.7093

NAAS Rating: 3.76

Big Data with Deep Learning (Supervised Vs. Unsupervised): A Survey

23

Figure 4: Multi-Layer Artificial Neural Network Many algorithms could be implemented to train the network either supervised or unsupervised. For example, back-propagation algorithm which is a supervised learning algorithm that uses a forward activation function which defines the output of each node according to a given input or set of inputs as shown in figure 5. It is a binary function results in either the neuron is firing or not [6].

Figure 5: The Activation Function for each Node.

SOME IMPORTANT FEATURES OF SUPERVISED AND UNSUPERVISED LEARNING ALGORITHMS CAN BE HIGHLIGHTED IN THE FOLLOWING WAY. In Supervised Learning Methods •

The Teacher is presented during learning process.

•

Every input pattern used to train the network.

•

The Learning process is based on computing actual output and correct output.

•

Error is used to change the parameters of the net.

In Un-Supervised Learning Method •

The teacher is not available.

•

The desired output is not presented to the network.

•

The system learns by its own by adapting the structural features in the input patterns.

The general role of supervised and unsupervised learning is shown in figure 6.

www.tjprc.org

editor@tjprc.org

24

Abeer Anwar Al-Qassab & Hala Jaffar Al-Lawati

Figure 6: Classification and Clustering.

IN SEMI-SUPERVISED LEARNING METHOD •

It is a middle ground between supervised and unsupervised learning.

•

Training an algorithm with a reward system.

•

Providing feedback agent performs the best action in a particular situation.

Deep Learning for Big Data Big Data has become important for many organizations (in both public and private sectors). These organizations are collecting massive amounts of domain-specific information, which can contain useful information for different problems such as national intelligence, cyber security, marketing, and medical informatics. Examples of these companies are Google and Microsoft. They are analyzing large volumes of data for business analysis and decisions, or for impacting existing and future technology.

So this amount of data needs a complex abstraction as data representations through a hierarchical learning process [7]. The term hierarchical learning here is referred to neural networks.

Deep learning is a form of machine learning that can utilize either supervised or unsupervised algorithms, or both.

In deep learning, multi-layer (with many hidden layers) neural network is being used as shown in figure 7. In which each layer takes input from the previous layer, processes it, and outputs it to the next layer, in a daisy-chain fashion.

Deep learning can be used to generate high level of abstraction. So for complex abstractions of data representations through a hierarchical learning process, deep learning models produce results faster than standard machine learning [4]. And also a deep learning model will learn the features that are important by itself, instead of requiring to be manually selecting the pertinent features.

Figure 7: Deep Neural Network for Deep Learning.

Impact Factor (JCC): 9.7093

NAAS Rating: 3.76

Big Data with Deep Learning (Supervised Vs. Unsupervised): A Survey

25

For a feed forward neural network, the number of hidden layers plus one is referred to by credit assignment path (CAP) depth of the network. A Deep Neural Network is denoted by (DNN) [9][10]. For those types of DNN in which data can flow in any direction and the CAP depth is potentially unlimited, then they are referred to as Recurrent Neural Networks (RNN), and they are used for applications such as language modeling [11].

The most modern deep learning models which are based on artificial neural networks is the Convolutional Neural Networks (CNN), which can include propositional formulas, and they are used for computer vision and speech recognition [12].

Supervised and Unsupervised Deep Learning towards Data Modeling Part of “Data Analyzing Process” Different learning algorithms can be applied to the structures of the Deep Neural Network, each of them has advantages and dis-advantages. They are classified as Supervised, unsupervised, and semi-supervised learning algorithms. These learning processes play an important role in constructing the data models in the modeling stage (of data science stages).

Supervised Learning Tasks Deep learning methods obviate feature engineering, by translating the data into compact intermediate representations akin to principal components, and derive layered structures that remove redundancy in representation.

This type of learning is responsible for the Recognition and Calcification of data models which represents an important part of the main steps of the Data Science analysis process, as shown in figure.8.

Many learning algorithms could be used for the learning process. They are implemented according to the type of the data and application [20].Some examples of these algorithms are: •

Back-Propagation: Computes the partial derivatives (or gradient) of a function. The function gradient at each iteration.

•

Stochastic Gradient Descent: An intuitive way to think of Gradient Descent is to imagine the path of a river originating from top of a mountain. The goal of gradient descent is exactly what the river strives to achieve.

•

Dropout: A very powerful machine learning systems with a large number of parameters.

Unsupervised Learning Tasks This is a special type of deep learning algorithm. The main task for this type of learning algorithm is Clustering. It’s job is labeling the new coming data. This is an important benefit because unlabeled data are more abundant than labeled data.

Examples of deep structures that can be trained in an unsupervised manner are neural history compressors [13].

This is a harder task than supervised learning. The role of this type of learning clearly appears in the part of Data Science analysis process which is prior to the training part, as shown in figure 8 Example of Unsupervised algorithms is: Unsupervised Pre-Training Algorithm •

Initialize hidden layers using unsupervised learning.

www.tjprc.org

editor@tjprc.org

26

Abeer Anwar Al-Qassab & Hala Jaffar Al-Lawati

•

Encourage hidden layers to encode that structure

Figure 8: The Role of ml in Data Modeling.

Table1 shows a simple comparison between supervised and unsupervised Deep learning.

Table 1: Supervised and Unsupervised Deep Learning Unsupervised Supervised Deep Learning Deep Learning Performing Initial training Next steps Stage Unlabeled Labeled dataset Type of Data dataset Varity & Filtered Size of Data abundant More complex Less Complex method and Execution method and less more execution time execution time time Provides unlabeled data Provides an and tries to “answer key”to make sense of be used to them by Operation evaluate its extracting accuracy on features and training data patterns on its own.

Most Common Applications that use Deep Learning Deep learning has recently seen a “surge” in popularity as a way to accelerate the solution of certain types of difficult computer problems, it is most notably available in the following applications: •

Automatic speech recognition: It is implemented by using LSTM RNNs which can learn "Very Deep Learning" tasks [14].

•

Image recognition: Implemented by using MNIST. It is composed of handwritten digits and includes 60,000 training examples and 10,000 test examples [15].

Impact Factor (JCC): 9.7093

NAAS Rating: 3.76

Big Data with Deep Learning (Supervised Vs. Unsupervised): A Survey

•

27

Natural language processing: Implemented by using LSTM which helped to improve machine translation and language modeling [15].

•

Drug discovery and toxicology: Implemented by using a large percentage of candidate drugs fail to win regulatory approval.

•

Customer relationship management: In which deep reinforcement learning has been used to approximate the value of possible direct marketing actions [16].

•

Image restoration: In which deep learning has been successfully applied to inverse problems such as de-noising, super-resolution, in-painting, and film colorization.

•

Financial fraud detection:In which deep learning is being successfully applied to financial fraud detection and anti-money laundering [17].

•

Commercial activity:

Examples of organizations that employ deep learning for particular applications are as follows: •

Facebook's AI: Tasks are performed automatically to make tagging uploaded pictures with the names of the people in them [18].

•

Google's DeepMind Technologies: These technologies developed a system capable of learning how to play Atari video games using only pixels as data input [19].

•

Many other organizations also employed deep learning for other tasks like mobile augmented reality applications, or provide a robot with the ability to learn new tasks through observation.

CONCLUSIONS Big data is a technology with the potential to extract hidden patterns and find better and more effective solutions for many problems. It transforms various industries and it has the positive impact on the business operation which is undeniable.Machine learning is the technology that finally makes many tangled processes that previously required a lot of time and effort routine operations that require mere supervision and not a manual action.Deep learning is a very important and effective learning process which works with big data. It represents a basic part of machine learning process. It uses deep neural network to represent the features and the different models of the input data.

The artificial neural network which is used in deep learning has different structures according to the data and the application, and these are DNN, CNN, and RNN. The learning process could be supervised which is used for recognizing and modeling the labeled data, and unsupervised which is used for clustering and labeling the input data. These learning method are playing a very important role of data analysis process which is denoted as “data modeling stage”. Other types of learning like semi-supervised which is a combination between the two types. It uses a small amount of labeled data bolstering a larger set of unlabeled data.

Deep learning is now being used as asolution of certain types of difficult computer problems. And it is adopted in many organizations.

REFERENCE www.tjprc.org

editor@tjprc.org

28

Abeer Anwar Al-Qassab & Hala Jaffar Al-Lawati 1.

“Artificial Intelligence Structures & strategies for complex problem solving”, George F. Luger, 6th edition, copyright 2016, Pearson.

2.

“Role of Data Science in Artificial Intelligence”, Karen Lin, Feb 19. https://towardsdatascience.com/role-of-data-science-inartificial-intelligence-950efedd2579).

3.

“Artificial intelligence vs. big data: Comparing emerging techs”, https://searchstorage.techtarget.com/feature/Artificialintelligence-vs-big-data-Comparing-emerging-techs.Robert Sheldon, Jan 23, 2018

4.

“Machine Learning, Deep Learning, and AI: What’s the Difference?” Hans-Dieter Wehle, Conference Paper · July 2017.

5.

“Machine Learning - Definition and application examples”, by Lina Klass, Aug 29, 2018, Spotlight Mental-the network for light metal casting.

6.

“Big Data Analytics Using Neural networks”, Chetan Sharma, Spring 2014, Master theses and graduate research, San Jose State University.

7.

“Deep learning applications and challenges in big data analytics”, Maryam M Najafabadi, Flavio Villanustre, Taghi M Khoshgoftaar, Naeem Seliya, Randall Wald and Edin Muharemagic Journal of Big Data (2015)

8.

“What is Big Data and Machine Learning 7 Real-World Applications”,

9.

"Learning Deep Architectures for AI" (PDF). Foundations and Trends in Machine Learning. Bengio, Yoshua (2009). 2 (1): 1– 127

10. "Deep Learning in Neural Networks: An Overview". Neural Networks. Schmidhuber, J. (2015). 61: 85–117.

11. "Sequence to Sequence Learning with Neural Networks" (PDF). Sutskever, L.; Vinyals, O.; Le, Q. (2014). Proc. NIPS. 12. "Deep convolutional neural networks for LVCSR". Sainath, T. N.; Mohamed, A. r; Kingsbury, B.; Ramabhadran, B. (May 2013). IEEE International Conference on Acoustics, Speech and Signal Processing: 8614–8618.

13. “Deep Learning”. Scholarpedia, Jürgen Schmidhuber (2015).

14. "Deep Learning in Neural Networks: An Overview". Neural Networks. 61: 85–117 Schmidhuber, J. (2015).

15. "MNIST handwritten digit database, Yann LeCun, Corinna Cortes and Chris Burges". yann.lecun.com.

16. Tkachenko, Yegor (April 8, 2015). "Autonomous CRM Control via CLV Approximation with Deep Reinforcement Learning in Discrete and Continuous Action Space".

17. . "Deep learning: the next frontier for money laundering detection". Global Banking and Finance Review. Czech, Tomasz.

18. "Facebook's 'Deep Learning' Guru Reveals the Future of AI". Metz, C. (12 December 2013).

19. "Google AI algorithm masters ancient game of Go". Nature News & Comment. Retrieved 2016-01-30.

20. “The 10 Deep Learning Methods AI Practitioners Need to Apply”, James Le, Nov 17, 2017.

21. “Big Data “, webopidia, Vangie Beal.

22. “What is big data? A consensual definition and a review of key research topics”, Andrea De Mauro, Marco Greco, and Michele Grimaldi, AIP Conference Proceedings 1644, 97 (2015); 23. “Big data and machine learning: thigs you need to know”, the appSolutions.

24. “Data science vs. machine learning”, Edureka’s Data Science Course on python, Published on Feb 6, 2019.

Impact Factor (JCC): 9.7093

NAAS Rating: 3.76

Big Data with Deep Learning (Supervised Vs. Unsupervised): A Survey

29

AUTHOR’S PROFILE

“Abeer Anwar Abdul-Rahman” The Faculty of (Information Technology, Al-Zahra College for Woman, Sultanate of Oman), Master Degree In Computer Science, Department of Computer Science/ College of Science/ Baghdad University/ Iraq. Full Time Lecturer in Al-Zahra College for Woman/ Oman (for 10 years).Lecturer Gulf College/ Oman (for 3 years).

Senior Software Developer in “Western Co. for IT/ Baghdad/ Iraq” (for 2 years). Lecturer in Baghdad University/ College of Science/ Computer Science Dept. (for 2 years). Software Developer in “IPA Center/ Baghdad/ Iraq” (for 2 years).

Participated in many conferences and have Publications.

“Hala JaffarAl-Lawati The Faculty of Information Technology, Al-Zahra College for Woman, Sultanate Of Oman, Master Degree In Computer Science, Department Of Computer Science, College Of Science Sultan Qaboos University, Muscat, Sultanate of Oman. 2006, April. From 2003 To Aug. 2007 Teaching and Researching Assistant Al-Zahra College for Woman, and Sep. 2007 Till Now Full Time Lecturer Al-Zahra College For Woman Sultanate Of Oman. Participated in many conferences and have Publications.

www.tjprc.org

editor@tjprc.org