Semantic Labs

Ease of access to information and ability to conveniently share the same has resulted in exponential rise in data which needs to stored, processed and retrieved within a constrained time limit. At the same time the generation and inflow of data continues to grow at a significant high rate. Under such circumstances, we desire a system which is easily and effectively scalable in real time. Scalable solutions need to take under consideration concerns not only related to managing large data but as well as to have distributed computing grid in place that is effective in delivering. The computational grid is made of low end cost effective hardware which scale and adapt linearly to any change in the system.

Bioinformatics is a vast field encapsulating gene expression, structural and functional proteomics, computational biology, DNA Sequencing, functional genomics, metabolic pathways and many such advanced areas of research. The primary challenge across all is to model them into mathematical or algorithmic problem and harness the computational power of computers to achieve the desired goals. A wide range of advanced algorithms for string matching, network topology, predictive statistical models, graph algorithms, clustering and image processing are used to assist in solving problems. Named Entity Identification and NLP techniques are as well used to process medical and pharmaceutical related literature to store, retrieve and extract desired data from large data stores. It also requires integration with several online ontologies and data sources to enrich results.

Personalized learning has always been a crowd puller but with the assistance of technology, it can take a totally different form. Think about analyzing a student behavior or his/her solution approach to a particular problem and recommending an intervention or a sub-topic, sequentially suitable to him/her and not what other fellow students are learning. Sophisticated “intelligent” algorithms can be applied to the data generated by different systems/applications to come up with a personalized recommendation.

The primary challenge in HR solutions is dealing with two complimentary data sets, one class belongs to job listing published by companies and other is the resume profiles submitted by individuals. For a job aggregator company, the primary task is to aggregate data from heterogeneous data sources. Since the count of job listing and the resume can be in tens of thousands, practically it is impossible to manually achieve the best association of job listing to resumes. A use of vector-space model and clustering algorithm effective reduces the problem space and assist to eliminate substantial large number of irrelevant matches with high accuracy. The same fundamentals can be extended to several other domains where primary obstacle is to map entities form complimentary data set.

SEO is intended to improve the search engines visibility towards web pages in an organic manner. On the contrary Adwords is a search marketing strategy provided by Google to facilitate explicit advertisement. Both techniques inherently require an ability to auto analyze large number of dynamic web pages and predict the core concepts underlying their content. Advanced statistical and Markov models assist in building NLP algorithms for auto generate key concepts. Another aspect at the core of SEO is linkage building across web pages ans also across website which in turn influences the relevance order in search engine results.

Social Networks has revolutionized the means of interaction and exchange of information among people. Any business can be substantially benefited by exploiting the strong virtual network of individuals who could be potential clients. A business can utilize the network not only to advertise itself through virtual connections but can even derive inferences and strong connection that would enable aggressively pursuing finite goals. An example of such a solution is recommendation of a product to a targeted audience based on their networked connections.

Every medium and large institute such as enterprise businesses, schools and hospitals that generate large amount of data in form of documents and reports require an electronic means of storing and retrieving of data. More importantly they need an easy way of searching for a desired historical documents. A wide array of technology can be put together in place depending on the problem and the complexity of the solution demanded. If the documents are generated on continuous basis, then the data can grow exponentially in quantity, this would demand a large scalable distributed storage system. Whereas if the demand is for effective search then semantic search capability can be plugged in into the solution. Domain specific knowledge can be integrated into the solutions to enrich user experience.

Books, music, movies and various other channels of entertainment going digital has in turn opened doors for intelligent solution to play an important role. It’s quite interesting to observe that human interaction on topics such as movies and music are prominently motivated by similarity in interest. Fascinating intelligent systems can be developed that can smartly recommend books, movies, music or gifts online by capturing user’s virtual behavioral pattern, historical information and their similarity with other users across online community. Recommendation engine lies at the core of several domain, applications and technology which inherently desires intelligent decision making capability.

Litigation solution is associated with electronic discovery dealing with exchange of information in electronic format. The source of data in this case come from wide range of electronic documents such as email, PDF, presentations, spread sheet, word and text document. The textual contents of these documents need to be analyzed using NLP/IR techniques to extract key concepts from the large collection of document and use this meta information to categorize relevance of the document in the case. The solution requires semantic and contextual analysis of large amount of data.

Language has been the means of communication for human kind for hundreds of years. This has resulted in generation of large volumes of information in form of literature, history, art, science and philosophy in hundreds of languages. With the whole world going digital and over the web there is a deepening need for digital devices such as computers and mobiles to understand human languages. The application of the same may vary from sentiment analysis, language translation, document summarization, Named Entity such as Person/Place/Company name identification, Key concept generation etc. Sophisticated graphical language models Conditional Random Fields (CRFs), Maximum Entropy Markov Models (MEMMs) are used for Part-of-Speech (POS) tagging and Noun-Verb phrase identification. Machine learning, data mining and AI components can be as used to facilitate solving natural Language processing problems.

At root of all intelligence lays an ability to learn from past experience and use the same to make decisions. In machine learning the fundamental philosophy is to provide machines with historical data to learn from and train them to make sophisticated decision similar to that of human capability. Though human brain can make intelligent circumstantial decision, it does fail to scale when number of parameters needed are large in number. Machine learning algorithms provide ability to build statistical model over sufficiently large data set that can assist human in decision making. Machine learning can be broadly classified in supervised and unsupervised learning models. There are wide ranges of learning algorithms used in practice such as Support Vector Machines (SVMs), Bayesian Classifies, decision trees, neural networks etc

Ontology is a representation of knowledge though concepts and relationship shared between the concepts. Ontology endeavors to represent a system or a domain to its completeness. The structured nature of an ontology allows computers to consume them more effectively. A system which is enlightened with domain knowledge through ontology can be used to derive complex inferences which are impossible to deliver otherwise. For an instance, different doctors may refer to a symptom as heart attack, cardiac arrest or myocardial dysfunctional. A solution which is boosted by medical ontology can easily identify that all the three refer to serious heart diseases. This “inferencing” capability can be chained in forward or backward directions to derive complex conclusion. There are several ontologies which are being developed by various research institutes and companies in the areas of medical science, patent classification, life sciences, retails marketing, word-net, concept-net that can be integrated in solution as per the requirement.

A distributed system can be developed using a cluster of nodes distributed over the intranet. To achieve high scalability NoSQL data stores can be developed using distributed solutions such HBase and HIVE which reside over Hadoop’s distributed file system(HDFS), MongoDB, Casandra or similar map based data store. To harness computational advantage of distributed systems, functional language paradigm of MapReduce can be utilized which breaks down task into smaller subtasks getting executed in parallel over nodes in the grid/cluster. Another benefit of using Hadoop based solution is it’s fail safe nature, which means there is no on failure of any node in the system over the productivity.

Information storage and retrieval is an age old problem. The basic demand is an ability to search through tens of thousands of documents for a phrase or a keyword very efficiently. Building such a system requires pre-processing of data and converting it to inverted document index. This data structure allows fast search through a large number of document and retrieving exact document of interest. Another challenge here is the order in which retrieved results are presented back to the user. For instance use searching for *The world* would intend to give more importance to the word *world* rather then to the word *the*. This intention needs to be captured statistical by using scoring algorithm and ordering according to their importance. The search experience can be further enhances by pruning the word to the root word and allowing synonym based searching. Advanced search capability can be achieved by using LSI (Latent Semantic Indexing) which support semantic search capability without using any ontology based meta information.

Irrespective of domain/technology the biggest challenge while solving any problems using computers is to map problems from domains such as finance/life sciences/entertainment to appropriate data structures such as strings, graphs and trees. A more crucial step then is to discover which of the wide array of advances algorithms can be effectively applied. For instance Geometric Invariants can be used for protein structure comparison and gene expression can be studied using String Prefix Trees. Being aware of algorithmic possibilities is at the core of developing world class solutions.

Data mining is a science of designing and developing algorithm for identifying patterns and deriving conclusion based on available data. Data mining algorithms are prominently classified into two categories: Classification and Clustering. Classification algorithms are decision making models which are built on labeled data set whereas clustering algorithms as name suggest identifies group of data points which are similar to each other. In both cases fundamental goal is to derive meaning from large data set.