Deep learning: An introduction to framework
Shivam Shah
Ijariit Journal2018, IJARIIT
visibility
…
description
3 pages
link
1 file
Deep learning also called hierarchical learning is part of a broader family of machine learning method based on the learning data representation .learning can be supervised, unsupervised or reinforcement there are many deep learning libraries nowadays. Libraries contain direct function by which we can import the library and we can directly perform the algorithm on the data .now days number of such type libraries are available with their available feature and benefits .we have to select an appropriate tool for performing a particular task is difficult to decide. This research represents the comparative analysis of deep learning libraries .the main libraries which I will compare is tensorflow, pytorch, theano and caffe the parameters for comparing the libraries are the adoption, dynamic and static graph definition, debugging, visualization and data parallelism.
Related papers
Comparative Study on the Deep Learning Algorithms
IJRASET, 2021
Deep Learning methods have paved the way for elevating the future technology that is capable of changing the world. In modern times, size of data is increasing with the level of application. Deep learning enables the huge dataset to process the highly optimized algorithms with high accuracy as well as within low time. The network architecture of deep learning works similar to human brain nerves. The network accepts the input dataset and convert the data into matrix form that passed through multiple layers in which, each layer upgrade the data to deliver the prediction or classification at the end. Researchers explored the numerous deep learning models that portrayed an inspiration for developers and benefitted in the field of voice recognition, language translation, image categorization, stock market prediction etc. The concern behind the model is to effectively resolve the numerous tasks which need to distributed representation and human intelligence. The highly advanced processors like CPU and GPU has too enhanced the deep learning application through fast matrix calculations and image processing. We will take the sample of wind dataset and used it for comparing the different Deep Neural Network (DNN) artificial algorithm.
Demystifying Deep Learning Frameworks- A Comparative Analysis
JOURNAL OF MECHANICS OF CONTINUA AND MATHEMATICAL SCIENCES, 2019
Deep learning is a rapidly growing field of machine learning which finds the application of its methods to provide solutions to numerous problems related to computer vision, speech recognition, natural language processing, and others. This paper gives a comparative analysis of the five deep learning tools on the grounds of training time and accuracy. Evaluation includes classifying digits from the MNIST data set making use of a fully connected neural network architecture (FCNN). Here we have selected five frameworks-Torch ,Deeplearning4j, TensorFlow, Caffe & Theano (with Keras), to evaluate their performance and accuracy. In order to enhance the comparison of the frameworks, the standard MNIST data set of handwritten digits was chosen for the classification task. When working with the data set, our goal was to identify the digits (0-9) using a fully connected neural network architecture. All computations were executed on a GPU. The key metrics addressed were training speed, classification speed, and accuracy.
An Overview Studying of Deep Learning
International Journal of Scientific Research in Science, Engineering and Technology, 2020
Deep learning is a subfield of machine learning however both drop under the broad category of artificial intelligence. Deep learning is what powers the most human-like artificial intelligence that consents computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. Deep learning is making major advances in solving problems hence categorized in wider section of artificial intelligence. The main advantage of Deep Learning is to create an artificial neural network that can learn and make intelligent decisions on its own and to process large numbers of features makes deep learning very powerful when dealing with unstructured data.
Deep Learning Application Pros and Cons Over Algorithm
Deep learning is a new area of machine learning research. Deep learning technology applies the nonlinear and advanced transformation of model abstraction into a large database. The latest development shows that deep learning in various fields and greatly contributed to artificial intelligence so far. This article reviews the contributions and new applications of deep learning. The main target of this review is to give the summarize points for scholars to have the analysis about applications and algorithms. Then review tries to investigate the main applications and uses algorithms. In addition, the advantages of using the method of deep learning and its hierarchical and nonlinear functioning are introduced and compared to traditional algorithms in common applications. The following three criteria should be taken into consideration when choosing the area of application. (1) expertise or knowledge of the author; (2) the successful application of deep learning technology has changed the field of application, such as voice recognition, chat robots, search technology and vision; and (3) deep learning can have a significant impact on the application domain and benefit from recent research with natural language and text processing, information recovery and multimodal information processing resulting from multitasking deep learning. This review provides a general overview of a new concept and the growing benefits and popularity of deep learning, which can help researchers and students interested in deep learning methods.
A Comprehensive Study of Deep Learning Architectures, Applications and Tools
The Deep learning architectures fall into the widespread family of machine learning algorithms that are based on the model of artificial neural network. Rapid advancements in the technological field during the last decade have provided many new possibilities to collect and maintain large amount of data. Deep Learning is considered as prominent field to process, analyze and generate patterns from such large amount of data that used in various domains including medical diagnosis, precision agriculture, education, market analysis, natural language processing, recommendation systems and several others. Without any human intervention, deep learning models are capable to produce appropriate results, which are equivalent, sometime even more superior even than human. This paper discusses the background of deep learning and its architectures, deep learning applications developed or proposed by various researchers pertaining to different domains and various deep learning tools.
IJERT-An Overview of Deep Learning
International Journal of Engineering Research and Technology (IJERT), 2021
https://www.ijert.org/an-overview-of-deep-learning https://www.ijert.org/research/an-overview-of-deep-learning-IJERTCONV9IS05064.pdf This paper informs overview and recent advances in Deep Learning. Deep Learning (DL) is the subjectivity of Machine Learning (ML) and Machine learning is the subclass of Artificial Intelligence (AI). Now a day's Deep Learning is one of the most advanced scientific research area in all domains. Deep learning is driving significant advancements across industries, enterprises, health care, retail and financial services, automotive and daily life also. In this data is processed via neural networks and thus machine works alike human does. So the methods of Deep learning creates world shattering in all areas especially Machine Learning. Machine Learning and Deep Learning technologies work on algorithms and programming that activates the computer to think like a human and take a decision like learn by example. Deep learning uses Machine learning technologies to get solution of problems and make decisions. Day by day a new deep learning technique comes into the market and it gives good performance i.e. solution to the problem. Since Deep Learning evolves vast and fastest growing part of Artificial Intelligence.
Machine Learning and Deep Learning
International Journal of Innovative Technology and Exploring Engineering, 2019
Now-a-days artificial intelligence has become an asset for engineering and experimental studies, just like statistics and calculus. Data science is a growing field for researchers and artificial intelligence, machine learning and deep learning are roots of it. This paper describes the relation between these roots of data science. There is a need of machine learning if any kind of analysis is to be performed. This study describes machine learning from the scratch. It also focuses on Deep Learning. Deep learning can also be known as new trend of machine learning. This paper gives a light on basic architecture of Deep learning. A comparative study of machine learning and deep learning is also given in the paper and allows researcher to have a broad view on these techniques so that they can understand which one will be preferable solution for a particular problem.
DEEP LEARNING
Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech. In this paper, main topics about deep learning have been covered. The relationship between artificial intelligence, machine learning and deep learning has been mentioned briefly. Detailed information about deep learning has been given, ie. History and future of deep learning. Artificial neural networks has been reviewed. The importance of GPU and deep learning in big data have been shown deeply. Using areas of deep learning have been explained. Benefits and weaknesses of deep learning have been covered. The informations about deep learning algorithms, libraries and tools have been given.
Review on Different Software Tools for Deep Learning
International Journal for Research in Applied Science & Engineering Technology (IJRASET), 2022
Deep Learning Applications are being applied in various domains in recent years. Training a deep learning model is a very time consuming task. But, many open source frameworks are available to simplify this task. In this review paper we have discussed the features of some popular open source software tools available for deep learning along with their advantages and disadvantages. Software tools discussed in this paper are Tensorflow, Keras, Pytorch, Microsoft Cognitive Toolkit (CNTK).
Deep Learning Architecture : A Literature Survey
2018
Deep learning is a sub field of machine learning. Learning can be of supervised, semi-supervised and unsupervised. There are different types of architectures for deep learning . In this paper we are giving an overview of different architectures that are widely used and their application area. Deep learning is applied in many areas such as image processing, speech recognition, data mining, natural language processing, social network filtering, machine translation, bioinformatics and drug design. IndexTerms Deep learning ;deep learning architecture; machine learning _________________________________________________________________________________________________________________
Shah Shivam; International Journal of Advance Research, Ideas and Innovations in Technology
ISSN: 2454-132X
Impact factor: 4.295
(Volume 4, Issue 6)
Available online at: www.ijariit.com
Deep learning: An introduction to framework
Shivam Shah
[email protected]
Parul Institute of Engineering and Technology,
Parul University, Waghodia, Gujarat
ABSTRACT
Deep learning also called hierarchical learning is part of a broader family of machine learning method based on the learning
data representation .learning can be supervised, unsupervised or reinforcement there are many deep learning libraries
nowadays. Libraries contain direct function by which we can import the library and we can directly perform the algorithm on
the data .now days number of such type libraries are available with their available feature and benefits .we have to select an
appropriate tool for performing a particular task is difficult to decide. This research represents the comparative analysis of deep
learning libraries .the main libraries which I will compare is tensorflow, pytorch, theano and caffe the parameters for comparing
the libraries are the adoption, dynamic and static graph definition, debugging, visualization and data parallelism.
Keywords— Deep learning, Deep learning tools, Theano, Caffe, Pytorch, Tensorflow
1. INTRODUCTION
Deep learning is part of machine learning for learning about, multiple levels of representation and abstraction to make sense data
such as image sound and text. It is set of the algorithm in machine learning which typically uses the artificial neural network to learn
in multiple levels corresponding different levels of abstraction.
Deep learning tool consists of:
(1) Conventional neural network
(2) Document classification
(3) Image segmentation
(4) ML algorithm libraries
(5) Model training
(6) Neural network modeling
(7) Self-learning
(8) Visualization
Conventional neural network: it involves the use of the deep artificial neural network to analyze visual imagery. It helps image
clustering by similarities and does image reorganization with the scene.
Document classification: deep learning allows the document classification algorithmically where the task involves assigning of
the document to one or several class which makes it easy and sort to manage the document categorized by image, text etc.
Image segmentation: another feature of deep learning contains it allows the division of image into separate pieces that cover it
allow the represent the image into something that can easily be analyzed.
ML algorithm libraries: deep learning is an open source machine learning algorithm libraries for everyone.
Model training: deep learning helps in model training that provides machine learning algorithm with training data to learn from.
Neural network modeling: another feature of deep learning is neural network modeling that allows the use of an artificial neural
network to forecast and predict the outcome based on available data using simple mathematical modeling.
Self-learning: deep learning involves the self and unsupervised feature learning.
Visualization: another feature of deep learning that allows the ability to representing the data in graph or diagram.
2. DEEP LEARNING TOOLS
The machine learning model changing continuously. The key to shifting toward running machine learning algorithm on another
platform that makes that run on another platform that makes the application more smarter and intelligent .by using the deep learning
we can solve the complex problem makes possible. There are multiple framework or libraries are available. Each framework built
for different purpose and different manner here.
© 2018, www.IJARIIT.com All Rights Reserved
Page | 5
Shah Shivam; International Journal of Advance Research, Ideas and Innovations in Technology
Four deep learning frameworks for comparisons are selected namely tensorflow, pytorch, theano and caffe based on several facts.
The entire four frameworks are freely available for any user.
Tensorflow: tensorflow is one of the best deep learning frameworks arguably use buy the tech giant company it is an open source
library developed by Google researcher team named Google brain. It is released on it is developed 2 years ago in November 2015.
currently, the stable version of tensorflow is 1.11.0 it is written in python, C++ and cuda .tensorflow support language such as the
python, C++ and r to create deep learning model with a wrapper library
Tensorflow consist of two tools that are widely used:
Tensorboard for the effective data visualization of network modeling and performance
Tensorflow for the deployment of a rapid algorithm while retaining the same architecture and api. It also provides integration
with another tensorflow model. Which different from conventional practice and can be extended to serve other model and data
type.
Pytorch: the torch is scientific computing framework that offers the wide support of machine learning it is lua-based deep learning
uses widely among industry such Facebook, Google etc. it employs the cuda with c/C++ library processing and basically made to
scale the production of building model and provide flexibility. As pytorch runs on python .means the basic understanding of python
anyone can build the model. This library builds by the Facebook artificial intelligence group it firstly released 2 years ago. The
stable version of pytorch is 1.0. It was written in python, C++ and cuda.
Caffe: caffe is deep learning framework that supports interface like the python, C++ and Matlab as well as it also supports command
line interface. It is known for the transportability, speed, and applicability to the Conventional neural network. Caffe is developed
by the University of California, Berkeley it is written in C++ the last stable version was 1.0.
Theano: Theano is machine learning and deep learning library for python programming. In theano, the computation is expressed
using the numpy which are compiled to run efficiently on cpu or Gpu architecture. Developed by the university of Montreal. It was
initially released 11 years ago in 2007the latest stable version is 1.0.3it was written in python ad cuda.
3. DEEP LEARNING LIBRERIES COMPERESION
3.1 Comparison Based on Platform
Tenorflow, theano, pytorch and caffe all the four tools are available free online and anyone can use it the tensorflow can be used in
python, c++, Java , and go and r while theano and pytorch can be used in only python language .caffe is written in C++ and can be
supported in python matlan and C++.
Software
Caffe
Pytorch
Software licence
BSD
BSD
Theano
Tensorflow
Table 1: General comparison of libraries
Open source Platform
Written in
Yes
Linux, Macos ,windows C++
Yes
Linux, Macos ,windows C++,python ,cuda
BSD
Yes
Cross platform
python
Apache 2.0
Yes
Linux, Macos ,windows, C++,python ,cuda
Android
Interface
Python, Matlab, C++
Python
python
python,c++, java,go , r
Table 2: Comparisons based on the functionality of the library
Software
Open MP
support
Open
CL support
Cuda
Support
Automatic
Has Pre-trained Recurrent neural network
model
and conventional
differentiation
Neural network
Yes
Yes
YES
Caffe
Yes
Under development
Yes
Pytorch
Yes
Yes
Yes
Theano
Yes
Yes
Yes
Tensorflow
No
Via separately maintainable
Package
Via separately maintainable
Package
On roadmap but already
supports SYCL
Yes
Yes
Yes
YES
Throuth Lesagen’s YES
Model
Yes
YES
3.2 Graphing
The tensorflow, caffe, and theano are the static graph framework while the pytorch is a dynamic graph framework. In the static
graph network, the graph is defined statically before the model can run all the communication with the outer world is performed via
an object which is substituted by external data at runtime. In the dynamic graphing it provides more flexibility of computation.py
torch allows the graphing process very smoothly.
3.3 Visualization
Tensorboard which is part of tensorflow is a good tool when it comes to visualization. While the pytorch uses the visdom it is not
complete but it is convenient to use. The caffe generate the protext file. Caffe itself has python implementation to visualize network
© 2018, www.IJARIIT.com All Rights Reserved
Page | 6
Shah Shivam; International Journal of Advance Research, Ideas and Innovations in Technology
located in caffe root or python folder or we can use digits from NVIDIA which is visualization tool.d3viz is extended theano printing
module to interactively visualize compute graph .instead of the static picture it creates an html file which can be open in web
browser.
3.4 Data parallelism
one of the biggest features that differentiates the pytorch from other framework is declarative data parallelism we can use the direct
function to wrap any module and it will be parallelized over the batch dimension in the other way tensorflow allows to every
operation to run on a specific device .in the caffe by the data Parallelism it allows to run the simple code on the multiple GPU. A
solver and net will be used for every GPU so the batch size is effectively multiplied by the number of GPUs. It will again create
single GPU training; reduce the batch size in the network definition accordingly. Theano has feature allow t use the multiple GPU
at the same time in one function the multiple GPU require the use of the GPU array backend so it will work correctly .for the data
Parallelism the probably batter using the platoons.
4. CONCLUSION
Using which type of framework will depend on the application .if we want large scale image classification and object detection then
caffe is good caffe has the largest quantity of pre trained computer vision model. it makes it one of the best frameworks for the
computer vision the tensorflow has the largest active community by far learning the tensorflow is very easy for the tensorflow many
moocs and tutorial are available on the internet it has the very nice visualizing capability with tensorboard. It is generally the numeric
computation library means we can implement many algorithms with the tensorflow. The theano framework online learning tools
are not available easily it also causes loss of momentum so the recent model appears more slowly. State of the art model is always
be found. pretrained model is can be slow to appear. Pytorch is similar to Numpy in the way that it manages computation but it
needs the strong GPU support similarity to Numpy. It also has the c programming language as a backend so it much faster than
native python library Numpy could be GPU accelerated but it doesn’t have strong GPU support pytorch is specifically tailored for
GPU functionality in python.
5. REFERENCES
[1] S. Bahrampour, N. Ramakrishnan, L. Schott, M. Shah, "Research and Technology Center", Robert Bosch LLC Comparative
Study of Deep Learning Software Frameworks.
[2] Ian J. Goodfellow, David Warde-Farley, Pascal Lamblin, Vincent Dumoulin, Mehdi Mirza, Razvan Pascanu, James Bergstra,
Frederic Bastien, Yoshua Bengio, "Departement dInformatique et de Recherche Operationelle", Universite de Montreal
Pylearn2: a machine learning research library.
[3] R. Collobert, Laurens van der Maaten, A. Joulin, "Facebook AI Research 1 Hacker Way Menlo Park CA 94025 / 770 Broadway
New York NY 10003", USA Torchnet: An Open-Source Platform for (Deep) Learning Research.
[4] Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, Trevor
Darrell, "SUBMITTED to ACM MULTIMEDIA 2014 OPEN SOURCE SOFTWARE COMPE-TITION UC Berkeley EECS
Berkeley", CA 94702 Caffe: Convolutional Architecture for Fast Feature Embedding.
[5] Introduction to Deep learning algorithm Deep learning tutorials,http://deeplearning.net/tutorial/.
[6] A beginner’s introduction to Deep Learning Article by Samvita, http://hyperverge.co/a-beginners-introduction-to-deeplearning/.
[7] Deep learning book Chapter 1 Introduction, http://www.deeplearningbook.org/contents/intro.html.
[8] Cuda-convnet High-performance C++/CUDA implementation of convo-lution of neural networks, [online] Available:
https://code.google.com/p/cuda-convnet/.
[9] Evaluation of Deep Learning Toolkits, https://github.com/zer0n/deepframeworks/performance.
BIOGRAPHY
Shivam Shah
B. Tech (Information Technology)
Parul Institute of Engineering and Technology, Parul University, Waghodia, Gujarat
© 2018, www.IJARIIT.com All Rights Reserved
Page | 7
Related papers
Strange and wonderful: Numbers through a new (material) lens
Cuneiform Digital Library Journal, 2024
América Latina: una región ausente en un orden internacional en crisis
América Latina: transiciones ¿hacia dónde? Informe anual 2022-23, 2022
Galileo's Break with the Venetian Woodcut
Galileo‘s Break with the Venetian Woodcut, 2011
A Review of the Business Models of the Digital Collaborative Economy: Digital and Digitally-Supported Commons and Open Data
H2020–ICT-2016-1 DECODE D2.1 Multidisciplinary Framework on Commons Collaborative Economy DEcentralised Citizens Owned Data Ecosystem D2.1 Multidisciplinary Framework on Commons Collaborative Economy
Entre el papel y la carne:Representaciones del cuerpo de las mujeres y las batallas por el dominio de su sexualidad. Un análisis desde la prensa escrita y la vida cotidiana en la región urbana josefina, 1960-2010
Entre el papel y la carne: Representaciones del cuerpo de las mujeres y las batallas por el dominio de su sexualidad. Un análisis desde la prensa escrita y la vida cotidiana en la región urbana josefina, 1960-2010, 2016
The effect of laughter Yoga on general health among nursing students
Iranian journal of nursing and midwifery research, 2014
“Identità dis-integrate: le narrazioni della psiche nella letteratura migrante italiana”
P. Bria, E. Caroppo, M. Colimberti, P. Brogna (a cura di), Trattato italiano di psichiatria interculturale e delle migrazioni, Roma, Società editrice Universo, 387-98, 2010
ISSN and DOI Digital Object Identifier Number
Digital Object Identifier Number, 2024
A year in review in Minerva Anestesiologica 2022: anesthesia, analgesia, and perioperative medicine
Minerva Anestesiologica
Optimalisasi Ventilasi Inlet Pada Ruang Sandblasting Menggunakan CFD (Computational Fluid Dynamics)
2020
A Preliminary Design of a Depthand Roll-Controllable Towed Vehicle for Ocean Measurements
Journal of the Society of Naval Architects of Japan, 1988
On Edge Detection
IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986
Study of the Structural, Optical, Electrical and Morphological Properties of Nickel Sulfide Thin Films Used in Supercapacitors
Annals of West University of Timisoara - Physics, 2021
An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era
arXiv (Cornell University), 2022
NRAS mutation causes a human autoimmune lymphoproliferative syndrome
Proceedings of the National Academy of Sciences, 2007