The last time we attended a Big Data meetup, we learnt about Real time analytics and machine learning. On the 15th of November 2017, we found ourselves once more at the Virtusa auditorium at Orion City for another edition of the Colombo Big Data meetup.
The first topic of the day or rather, the evening at the Colombo Big Data Meetup was Linguistic analysis in data warehousing. The session was conducted by Dinesh Asanka. Dinesh, a visiting lecturer at SLIIT, spoke about why linguistic analysis is needed. He started off by defining what linguistics is.
According to Dinesh Asanka, Linguistics is the scientific or systematic study of language. We essentially use language to communicate our ideas to someone else. We use various terms to enhance the base value of somebody or something. In languages there are different rules. Some are historical, some are scientific.
He then spoke about typical linguistic terms such as low medium and high which are used to measure a given dataset. These can be enhanced by adding terms such as not low, very low, low medium, high medium etc. You can even combine them to create terms such as “low and not high” etc.
Dinesh’s next topic at the Colombo Big Data Meetup was about a data warehouse. A data warehouse is an enterprise wide framework. A framework that can be used to build your own customized platform for it. This is not only a place to store all our data, but also it’s about comprehensive technology. You take data from sources, filter said data and then load it to your data warehouse. From there, you can carry out numerous data analytics.
If you’re analyzing sales, there’s no set path to analyze it. That’s where the framework comes into play. It gives you a choice of parameters to use. In modern terms, data can be anything. Be it a tweet, a Facebook status, a video file, an audio file, almost anything can be perceived as data.
There are limitations in a traditional data warehouse. When you want to analyze something, you must first label it. For example, if you want to find the age groups of a given dataset, you would give them labels. That is called the Bucketized method. With these analytics, you can generate a report.
Asanka’s next topic was about fuzzy theory. In mathematics, fuzzy sets are sets whose elements have degrees of membership. Essentially this is where every object should have a weightage. The idea is to get some ideas of fuzzy into warehousing. Using an example of fuzzy theory, Asanka also spoke about Fuzzy membership and how data has a weightage and not just a label.
Asanka also touched on various membership functions in Fuzzy theory. But where does linguistics come in? In Fuzzy theory, there are a number of operations such as concentration, Fuzzification, Simple operations etc., all of which Asanka spoke about.
So in essence, if you used a traditional warehouse, you would only get small range descriptions, but with Linguistic Analysis, you would have a number states to show a more detailed report with greater analysis. This effort is to bring in linguistic variables to give greater understanding to your data to make more informed decisions.
In case you don’t know what Azure HDInsight is, it is a product of Microsoft that deals with Big Data analytics. In essence, Azure HDInsight is a managed Open Source Big Data analytics service for the enterprise. Priyankara spoke about Hadoop clusters and how to create a Hadoop cluster within a matter of minutes using Azure HDInsight.
Well, Priyankara explains it as a scalable fault tolerant open source programming framework for distributed processing and distributed storage for large datasets on commodity hardware as a cluster. Originally conceptualized by Google, the Hadoop concept was developed by Yahoo. Priyankara also spoke about why we would need to carry out instructions across multiple machines or environments.
Getting into slightly more technical terms, Priyankara explained that Hadoop is a cluster configured with a minimum of two servers: a name mode and worker nodes. You can add as many worker nodes as you want to distribute the workload. He then spoke of the three components in a Hadoop system. These are HDFS (Hadoop Distributed File System), MapReduce, and YARN (Yet another resource negotiator).
Priyankara then went on to speak about his topic: Azure HDInsight. This is essentially Apache Hadoop running on Microsoft Azure. The key advantage is that a Hadoop system can be setup within 15-20 minutes.
Azure HDInsight is fully managed and completely open source. Priyankara also spoke about HortonWorks. This is a big data software company that develops and supports Apache Hadoop, for the distributed processing of large data sets across computer clusters. If a Hadoop update appears, you don’t need to worry. HortonWorks will update everything accordingly and all the latest features of Hadoop will be made available on Azure HDInsight.
Priyankara then went into detail about how a Hadoop cluster is set up and the process involved. During this, he also spoke about Azure HDInsight Cluster types such as Apache Hadoop, Apache spark, Apache Storm, Apache Hbase, Interactive Hive, Microsoft R Server etc. Priyankara also spoke about ome vital settings such as Name and Subscription, Cluster type, OS, HDinsight version, Cluster tier, Resource group etc.
With a few more examples and demonstrations, Dinesh Priyankara’s session came to an end. In closing, he emphasized that the community is still growing and h encouraged those attending to feel free to sign up to give a presentation at the next meet up. And thus, latest edition of the Colombo Big Data meetup came to an end.
Subscribe to our mailing list and get interesting stuff and updates to your email inbox.
Workshop Agenda - The main purpose of the workshop is to give students the ability to analyze and present data by using Azure Machine Learning, and to provide an introduction to
Workshop Agenda –
The main purpose of the workshop is to give students the ability to analyze and present data by using Azure Machine Learning, and to provide an introduction to the use of machine learning and big data.
Module 1: Introduction to Machine Learning
This module introduces machine learning and discussed how algorithms and languages are used.
· What is machine learning?
· Introduction to machine learning algorithms
· Introduction to machine learning languages
Module 2: Introduction to Azure Machine Learning
Describe the purpose of Azure Machine Learning, and list the main features of Azure Machine Learning Studio.
· Azure machine learning overview
· Introduction to Azure machine learning studio
· Developing and hosting Azure machine learning applications
Module 3: Managing Datasets
At the end of this module, the student will be able to explore various types of data in Azure machine learning.
· Categorizing your data
· Importing data to Azure machine learning
· Exploring and transforming data into Azure machine learning
Module 4: Building Azure Machine Learning Models
This module describes how to use regression algorithms and neural networks with Azure machine learning.
· Azure machine learning workflows
· Using regression algorithms
· Using neural networks
Module 5: Using Azure Machine Learning Models
This module explores how to provide end users with Azure machine learning services, and how to share data generated from Azure machine learning models.
· Deploying and publishing models
· Consuming Experiments
Module 6: Using Cognitive Services
This module introduces the cognitive services APIs for text and image processing to create a recommendation application, and describes the use of neural networks with Azure machine learning.
· Cognitive services overview
· Processing language
· Processing images and video
· Recommending products
Register URL –
FB Event page-
(Tuesday) 12:00 am - 11:59 pm
Blue Chip Training0716092918
Discover new dimensions in connecting the Internet of Things with Narrowband IoT technology at the NB-IoT Forum and Hackathon, organized by Mobitel. Date - 23rd March 2018 at Trace Expert City
Discover new dimensions in connecting the Internet of Things with Narrowband IoT technology at the NB-IoT Forum and Hackathon, organized by Mobitel.
Date – 23rd March 2018 at Trace Expert City – Colombo 10.
Entrance – Free for a limited number of participants.
Register now – https://goo.gl/3cRdHJ
(Friday) 9:00 am - 5:00 pm
Trace Expert City
Maradana Rd, Colombo
Tech Coders V1.0 will be an online 12-hour problem solving competition. During this 12-hour period your problem solving skills will be put to the test through a series of questions. Competition
Tech Coders V1.0 will be an online 12-hour problem solving competition. During this 12-hour period your problem solving skills will be put to the test through a series of questions.
Competition will be conducted on HackerRank.
*Please note that you will be given access to the contest on HackerRank only if you fill this form on or before 11.59 pm on 22nd March (Thursday).
Organized by : Tech Seekers – Sri Lankan Community
24 (Saturday) 8:00 pm - 25 (Sunday) 8:00 am
One on one talk , Q&A and networking session with Manju Nishshanka,Founder and CEO,KRMG Capital. Mr.Nishshanka is a serial entrepreneur with extensive experience in financial markets and disruptive technologies.He has founded
One on one talk , Q&A and networking session with Manju Nishshanka,Founder and CEO,KRMG Capital.
Mr.Nishshanka is a serial entrepreneur with extensive experience in financial markets and disruptive technologies.He has founded and invested in several successful startups in fintech,blockchain ,AI,AR & VR and social media sectors.
He is a keynote speaker at the Digital Asset Investment Forum (DAIF 2018) and also serves on the board of NYU stern Blockchain and digital asset forum.
KRMG capital is an investment and advisory firm focused on early and growth staged startups and digital assets.
University of Sri Jayawardenepura,Sri Lanka partnered with KRMG capital to establish the first ever blockchain laboratory in Sri Lanka.
This will be an eyeopener to the vast potential and opportunities in Blockchain,Digital assets and Cryptonomics.
All are welcome to join the event and the networking session.
(Wednesday) 6:30 pm - 8:30 pm
Lakshman Kadirgamar Institute
Horton Place, Colombo 00700
Chandimal alahakoon 077 22 44 905
Subscribe to our mailing list and get interesting stuff and updates to your email inbox.