Background/Problem Statement

Recent successful AI techniques, particularly those based on deep learning,  are not yet adequate for more demanding applications. In particular, they require a very large amount of labelled training data, especially for more complex problems. One reason for this is that the methods do not utilise available knowledge well in combination with the learning methods.

Solution

We tackle the issue by embedding knowledge into the structure of the learning model. In particular, we develop methods for utilising two types of knowledge in order to empower deep learning for more challenging tasks: (i) knowledge about model structures, constraints, rules and facts, together with (ii) knowledge about inference and planning algorithms.

#1 Probabilistic graphical models provide a general method for specifying known constraints, rules, and dependencies. However, they usually do not utilise the powerful approximation capabilities of modern deep learning well. We provide a method for combining the capabilities of probabilistic graphical models and modern graph neural networks through the Factor Graph Neural Networks (FGNN).

#2 Temporal graphical models can be used for sequence prediction problems such as time series prediction, including stock prices and language models. Powerful inference methods, including particle filters have been developed for them. However, they typically do not utilise the powerful approximation capabilities of deep learning sequence prediction methods such as recurrent neural networks. We provide a method for exploiting knowledge of the particle filtering algorithm in recurrent neural networks  through the Particle Filter Recurrent Neural Network (PF-RNN). Experiments demonstrate that the proposed PF-RNNs outperform the corresponding standard gated RNNs on a synthetic robot localization dataset and ten real-world sequence prediction datasets for text classification, stock price prediction etc.

#3 Knowledge of dependencies among different components of a problem can often be nicely captured using graph neural networks and attention networks. We exploit graph neural networks and attention networks to learn to do effective feature matching in computer vision, to learn to optimise for the multiple travelling salesproblem, and to do aspect based sentiment analysis.

Benefits

The work advances the state-of-the-art in AI, enabling improved performance in sequence learning, natural language processing, computer vision, and combinatorial optimization.

Potential Application(s)

Computer vision, natural language processing, combinatorial optimisation problems

We welcome interest from the industry for collaboration/ co-development / customisation of the technology into a new product or service. If you have any enquiries or are keen to collaborate, please contact us.