- Dm-meeting - mailman.cs.vt.edu

Preliminary exam: Bijaya Adhikari Torg 3160A, 12pm, TODAY
by B. Aditya Prakash 07 May '19

07 May '19

Hello everyone, Sorry for the late notice: if you are free, please join us for Bijaya Adhikari's proposal (prelim exam) TODAY May 7 12pm at Torg 3160A. You can also join through zoom @ https://virginiatech.zoom.us/j/578961900 <https://www.google.com/url?q=https%3A%2F%2Fvirginiatech.zoom.us%2Fj%2F57896…> Meeting ID: 578 961 900 Title+abstract are below. Best, - Aditya ================== ***Title*** Domain-based Frameworks and Embeddings for Dynamics over Networks *** Abstract *** Broadly this thesis looks into network and time-series mining problems pertaining to dynamics over networks in various domains. Which locations and staff should we monitor in order to detect C. Difficle outbreaks in hospitals? How do we predict the peak intensity of the influenza incidence in an interpretable fashion? How do we infer the states of all nodes in a critical infrastructure network where failures have occurred? Leveraging domain-based information should make it is possible to answer these questions. However, several new challenges arise, such as (a) presence of more complex dynamics. The dynamics over networks that we consider are complex. For example, C. Difficle spreads via both people-to-people and surface-to-people interactions and correlations between failures in critical infrastructures go beyond the network structure and depend on the geography as well. Traditional approaches either rely on models like Susceptible Infectious (SI) and Independent Cascade (IC) which are too restrictive because they focus only on single pathways or do not incorporate the model at all, resulting in sub-optimality. (b) handling data sparsity. Additionally, data sparsity still persists in this space. Specifically, it is difficult to collect exact state of each node in the network as it is high-dimensional and difficult to directly sample from. (c) lack of generalizability. In many situations, the dynamics depend on a mixture of several models or are unknown. In such cases, methods which generalize well to unobserved (or unknown) models are required. Current approaches often fail in tackling these challenges as they either rely on restrictive models, or require large volume of data. In this thesis, we propose to leverage domain-based frameworks, which include novel models and analysis techniques, and domain-based low dimensional representation learning to tackle the challenges above for networks and time-series mining tasks. By developing novel frameworks, we can capture the complex dynamics accurately and analyze them more efficiently. For example, to detect C. Difficle outbreaks in a hospital setting, we use a two-mode model to capture multiple pathways of outbreaks and discrete lattice-based optimization framework. Similarly, we propose an information theoretic framework which includes geographically correlated failures in critical infrastructure networks to infer network states. Moreover, as we use more realistic frameworks to accurately capture and analyze the mechanistic processes themselves, our approaches are effective even with sparse data. At the same time, learning low-dimensional domain-aware embeddings capture domain specific properties (like incidence-based similarity between historical influenza seasons) more efficiently from sparse data, which is useful for subsequent tasks. Similarly, since domain-aware embeddings capture the model information directly from the data without any modeling assumptions, they generalize better to new models. Our domain-aware frameworks and embeddings enable many applications in critical domains. For example, our domain-aware frameworks for C. Difficle allows different monitoring rates for people and locations, thus detecting more than 95% of outbreaks. Similarly, our framework for product recommendation in E-commerce for queries with sparse engagement data results in a 34% improvement over the current Walmart.com search engine. Similarly, our novel framework leads to a near optimal algorithms, with additive approximation guarantee, for inferring network states given partial observation of the failures in networks. By exploiting domain-aware embeddings, we outperform non-trivial competitors by upto 40% for influenza forecasting. Similarly, domain-aware representations of subgraphs helped us outperform non-trivial baselines by upto 68% in graph classification tasks. *** Committee **** B. Aditya Prakash (chair) Madhav Marathe Naren Ramakrishnan Chandan Reddy Jimeng Sun (Georgia Tech)

1 0

Monday, May 6th, 2019 - 11 AM @ McBryde 133C, Bijaya Adhikari
by Alexander Rodriguez 05 May '19

05 May '19

Please join us for our next DM-Meeting. ****DM-Meeting Spring 2019**** WHO: Bijaya Adhikari WHEN: Monday, May 6th, 2019 - 11 AM WHERE: McBryde 133C WHAT: Bijaya will practice his preliminary proposal "" Title: Domain-based Frameworks and Embeddings for Dynamics over Networks Abstract: Broadly this thesis looks into network and time-series mining problems pertaining to dynamics over networks in various domains. We answer questions like which locations and staff should we monitor in order to detect *C. Difficle* outbreaks in hospitals as quickly as possible? How do we predict the peak intensity of the influenza incidence in the current season in an interpretable fashion? How do we select the group of queries which are most critical to ensure high user satisfaction for an E-commerce system? Due to ease of data collection in various domains, it is now possible to answer these questions. However, these questions pose several new challenges. The first challenge is to (a) incorporate more realistic dynamics. The dynamical processes in the domains we consider are complicated. For example, *C. Difficle* spreads via both people-to-people and surface-to-people interactions and in E-commerce users can decide to continue interaction with the system or arbitrarily decide to stop and move on. Traditional transmission models like SI, IC, and SIR are too restrictive to capture these processes. Hence, we need to incorporate more realistic dynamics into account to accurately answer the questions above. The second challenge is to (b) handle data sparsity. Despite data collection being easier and cheaper, data sparsity problems persist. Specially, the data related to incidence like infections and node failures in infrastructure networks are difficult to obtain as they are constrained by actual events and are hence hard to observe and directly sample from. The third challenge is to (c) ensure robust performance. For the questions presented above, the existing approaches are based on heuristics and fail to provide robust guarantees in performance. However, robustness is of much importance for the critical problems we study. In this thesis, we propose to leverage domain-based frameworks, which includes novel models and analysis techniques, and domain-based low dimensional representation learning to tackle the challenges above for networks and time-series mining tasks. The first challenge we must overcome is to incorporate realistic transmission models to capture the complicated dynamics. However, more realistic transmission model often requires more challenging analysis techniques in formulating problems and designing algorithms. Hence, we propose to leverage domain-based frameworks, including both the models and analysis techniques to answer tackle these problems. For example, for *C. Difficle* outbreak detection problem, we use two-mode disease model to capture both people-to-people and locations-to-people infection pathways of *C. Difficle* and discrete lattice based optimization framework. Similarly, we propose randomized user navigation model along with submodular function optimization for critical queries mining in e-commerce and leverage geographically correlated failure models with Minimum Description Length formulation to infer network states. We overcome the second challenge on data sparsity by leveraging novel models as well as by including domain-aware low dimensional embeddings. As we use more realistic models to accurately capture the dynamics, we can generalize from the sparse data better, hence overcoming the data sparsity problem. On the other hand, domain-aware embeddings capture the important domain-based aspect of the data and can help guide the learning architecture to infer the correct function with less data. We carefully construct domain-based frameworks by incorporating realistic models and most relevant analysis techniques necessitated by the model. This approach leads to meaningful problem formulation and often leads to near optimal algorithms. On the other hand, domain-based embeddings augment learning architectures and improve the performance. Our thesis has several applications in critical tasks in variety of domains. For example, our domain-aware frameworks for *C. Difficle*allows different monitoring rates for people and locations, which the prior works fail to do. Hence, our approach can detect more than 95% of outbreaks, which is greater than any other competitor. Similarly, our framework for product recommendation in E-commerce for queries with sparse engagement data results in 34% improvement over Walmart.com search engine. It also helped us in detecting most critical queries in e-commerce outperforming non-trivial baselines like page-ranks, most frequent queries, and so on. Similarly, we propose near optimal algorithms for inferring network states given partial observation of the failures in networks. On the other hand, by exploiting domain aware low dimensional representations, we outperform non-trivial competitors by up to 40 percentage for influenza forecasting. Similarly, domain-aware representations of subgraphs helped us outperform non-trivial baselines in the graph classification task. Best regards, *Alexander Rodriguez* PhD Student Department of Computer Science Virginia Tech _______________________________________________ Seminar website: *http://people.cs.vt.edu/~arodriguez/DM-Meeting/home.html <http://people.cs.vt.edu/~arodriguez/DM-Meeting/home.html>* Please contact Alexander Rodriguez (arodriguez(a)cs.vt.edu <esorour(a)cs.vt.edu>), if interested in giving a talk. Dm-meeting mailing list Dm-meeting(a)cs.vt.edu https://mailman.cs.vt.edu/mailman/listinfo/dm-meeting

1 1

Congratulations to Sorour and Xinfeng
by B. Aditya Prakash 26 Apr '19

26 Apr '19

Congratulations Dr. Amiri, a great job at the defense! Congratulations to Xinfeng for both finishing up his MS in CS, and also winning the department's best MS thesis award! (award will be given next week on May 1 during the dept's annual function, I will miss it, others in town should try to go!) Thanks to everyone who helped with the numerous practice talks :-) cheers, - Aditya

3 2

Tuesday, April 30th @ 1 PM, Tor 3160A, Bijaya Adhikari
by Alexander Rodriguez 21 Apr '19

21 Apr '19

Please join us for our next DM-Meeting. ****DM-Meeting Spring 2019**** WHO: Bijaya Adhikari WHEN: Tuesday, April 30th @ 1 PM WHERE: Tor 3160A WHAT: Bijaya will practice his SDM talk "" Title: Joint Post and Link-level Influence Modeling on Social Media Abstract: Microblogging websites, like Twitter and Weibo, are used by billions of people to create and spread information. This activity depends on various factors such as the friendship links between users, their topic interests and social influence between them. Social influence can be thought of as a latent factor, that may alter users posting and linking behaviors. Making sense of these behaviors is very important for fully understanding and utilizing these platforms. Most prior work in this space either ignores the effect of social influence, or considers its effect only on link formation or post generation. In contrast, we propose PoLIM, leveraging simple weak supervision, a novel model which jointly models the effect of influence on both link and post generation. We also give PoLIM-FIT, an efficient parallel inference algorithm which scales to large datasets. In our experiments on a large tweets corpus, we detect meaningful topical communities, celebrities, as well as the influence strengths patterns among them. Further, we find that there are significant portions of posts and links that are caused by influence, and this portion increases when the data focuses on a specific event. We also show that differentiating and identifying these influenced content benefits other specific quantitative downstream tasks as well, like predicting future tweets and link formation, where we significantly outperform state-of-the-art. Best regards, *Alexander Rodriguez* PhD Student Department of Computer Science Virginia Tech _______________________________________________ Seminar website: *http://people.cs.vt.edu/~arodriguez/DM-Meeting/home.html <http://people.cs.vt.edu/~arodriguez/DM-Meeting/home.html>* Please contact Alexander Rodriguez (arodriguez(a)cs.vt.edu <esorour(a)cs.vt.edu>), if interested in giving a talk. Dm-meeting mailing list Dm-meeting(a)cs.vt.edu https://mailman.cs.vt.edu/mailman/listinfo/dm-meeting

1 0

Tuesday, April 30th @ 1 PM, Tor 3160A, Bijaya Adhikari
by Alexander Rodriguez 21 Apr '19

21 Apr '19

Please join us for our next DM-Meeting. ****DM-Meeting Spring 2019**** WHO: Bijaya Adhikari WHEN: Tuesday, April 30th @ 1 PM WHERE: Tor 3160A WHAT: Bijaya will practice his SDM talk "" Title: Joint Post and Link-level Influence Modeling on Social Media Abstract: Microblogging websites, like Twitter and Weibo, are used by billions of people to create and spread information. This activity depends on various factors such as the friendship links between users, their topic interests and social influence between them. Social influence can be thought of as a latent factor, that may alter users posting and linking behaviors. Making sense of these behaviors is very important for fully understanding and utilizing these platforms. Most prior work in this space either ignores the effect of social influence, or considers its effect only on link formation or post generation. In contrast, we propose PoLIM, leveraging simple weak supervision, a novel model which jointly models the effect of influence on both link and post generation. We also give PoLIM-FIT, an efficient parallel inference algorithm which scales to large datasets. In our experiments on a large tweets corpus, we detect meaningful topical communities, celebrities, as well as the influence strengths patterns among them. Further, we find that there are significant portions of posts and links that are caused by influence, and this portion increases when the data focuses on a specific event. We also show that differentiating and identifying these influenced content benefits other specific quantitative downstream tasks as well, like predicting future tweets and link formation, where we significantly outperform state-of-the-art. Best regards, *Alexander Rodriguez* PhD Student Department of Computer Science Virginia Tech _______________________________________________ Seminar website: *http://people.cs.vt.edu/~arodriguez/DM-Meeting/home.html <http://people.cs.vt.edu/~arodriguez/DM-Meeting/home.html>* Please contact Alexander Rodriguez (arodriguez(a)cs.vt.edu <esorour(a)cs.vt.edu>), if interested in giving a talk. Dm-meeting mailing list Dm-meeting(a)cs.vt.edu https://mailman.cs.vt.edu/mailman/listinfo/dm-meeting

1 0

Monday, April 22th @ 11 AM, McB 133C, Sorour Amiri
by Alexander Rodriguez 16 Apr '19

16 Apr '19

Please join us for our next DM-Meeting. ****DM-Meeting Spring 2019**** WHO: Sorour Amiri WHEN: Monday, April 22th @ 11 AM WHERE: McB 133C WHAT: Sorour will practice her PhD defense "" Title: Task-specific Summarization of Networks: Optimization and Learning Abstract: Networks (also known as graphs) are everywhere. People-contact networks, social networks, email communication networks, internet networks (among others) are examples of graphs in our daily life. The increasing size of these networks and simplicity to collect data diminishes our understanding of patterns in data. It is essential and at the same time challenging to analyze and understand the networks. Consider a social network such as Facebook or Twitter. A graph can represent the friendship between users. If we take the profile information or the strength of the friendship of users into account, we can have a weighted attributed graph. Also, we can consider the information associated with time stamps and produce a temporal friendship network. Mining and summarizing these graphs is an important task which reveals key patterns in the networks and helps in many applications such as visualizing and highlighting different communities or influential users in the network. In addition, summarizing networks helps in accelerating existing graph algorithms. We can run expensive algorithms such as ``finding influential users" or ``generating high-quality graph layouts" on smaller summary graphs and map the results to the original graph later. Finally, summarizing networks provides a high-level abstraction of them and helps in making generalized methods for a set of similar graphs. It is notable that different summarizes are desired for different purposes. For example, to stop viral infections, one may want to find an effective policy to immunize people in a people-contact network. In this case, a high-quality network summary should highlight the best people to immunize. Others may want to detect communities in the same people-contact network. Hence, we need another summary to show the communities. This means that for each task, we must design a specific method to reveal the related patterns. Despite the importance of summarizing networks for particular tasks, there has not been much work in this area. In this thesis, we design task-specific summarization frameworks for univariate and multivariate networks to reveal their important patterns and to accelerate existing algorithms and making sense of them. We start with optimization-based approaches to summarize graphs for a particular task and finally propose general frameworks which learn how to summarize the network for a given task and generalize it to similar networks. *Optimization-based approach:* Given a large network and a task, we propose summarization algorithms to highlight the characteristics of the graph (i.e., structure, attribute, label, dynamic) with respect to the task. We develop effective and efficient algorithms for summarizing univariate and multivariate graphs given various tasks such as content-aware influence maximization and time segmentation. In addition, we study many real-world networks and their summary graphs such as people-contact, news-blogs, etc. and visualize them to make sense of their characteristics given the input task. *Learning-based approach:* As our next step toward graph summarization, we propose a unified framework which learns the process of summarization for a given task. First, we design a generative algorithm to learn to summarize graphs for any Graph Optimization Problem (GOP). Next, we go further and add sparse human feedback to the learning process of summarization for the given optimization task. Our models and frameworks lead to meaningful discoveries. We also solve problems from various domains such as epidemiology, marketing, social media, cybersecurity, and interactive visualization. Best regards, *Alexander Rodriguez* PhD Student Department of Computer Science Virginia Tech _______________________________________________ Seminar website: *http://people.cs.vt.edu/~arodriguez/DM-Meeting/home.html <http://people.cs.vt.edu/~arodriguez/DM-Meeting/home.html>* Please contact Alexander Rodriguez (arodriguez(a)cs.vt.edu <esorour(a)cs.vt.edu>), if interested in giving a talk. Dm-meeting mailing list Dm-meeting(a)cs.vt.edu https://mailman.cs.vt.edu/mailman/listinfo/dm-meeting

1 0

PyTorch-BigGraph: Faster embeddings of large graphs
by B. Aditya Prakash 02 Apr '19

02 Apr '19

https://ai.facebook.com/blog/open-sourcing-pytorch-biggraph-for-faster-embe… Looks interesting! Apologies for the brevity Sent from my iPhone

1 0

Fwd: [Dac-faculty] Fwd: ECE- FACULTY FACULTY Faculty candidate seminar - Machine Perception for Autonomous Systems
by B. Aditya Prakash 26 Mar '19

26 Mar '19

Seems interesting! You guys should attend. Btw, also this week there are talks by Jen Neville and Mike Jordan; you should definitely attend those. There is a student meeting with Jen too; I would suggest you try to make it for that too, if possible. Cheers, - Aditya Apologies for the brevity Sent from my iPhone Begin forwarded message: > From: Jia-Bin Huang via Dac-faculty <dac-faculty(a)cs.vt.edu> > Date: March 26, 2019 at 7:01:27 PM EDT > To: "dac-faculty(a)cs.vt.edu" <dac-faculty(a)cs.vt.edu> > Subject: [Dac-faculty] Fwd: ECE- FACULTY FACULTY Faculty candidate seminar - Machine Perception for Autonomous Systems > Reply-To: Jia-Bin Huang <jbhuang(a)vt.edu> > > Dear Colleagus, > > I will be hosting a faculty seminar for the ECE machine perception position. As the candiate’s research overlaps with several of the current DAC faculty members (machine learning, social networks, data mining, graph signal processing), I am forwarding the talk information and hope to see you and your students could attend the talk. > > Time: 10:00 AM - 11:00 AM March 28 > Location: 457B Whittemore > > Thank you very much! > > Best, > > Jia-Bin > > ---------- Forwarded message --------- > From: Jia-Bin Huang <jbhuang(a)vt.edu> > Date: Tue, Mar 26, 2019 at 12:00 PM > Subject: ECE- FACULTY FACULTY Faculty candidate seminar - Machine Perception for Autonomous Systems > To: <ece-faculty-g(a)vt.edu> > > > Dear Colleagues, > > You are invited to a presentation this Thursday on the following topic by Dr. Siheng Chen (Research Scientist at MERL). > > Time: 10:00 AM - 11:00 AM March 28 > Location: 457B Whittemore > > Title: Data science with graphs: From social network analysis to autonomous driving > > Abstract: A massive amount of data is being generated at an unprecedented level from a diversity of sources, including online social networks, cyber-physical systems, biological networks, urban traffic networks, 3D point clouds, and many others. These seemingly different types of data share a similar pattern: data are associated with complex and irregular structures. The necessity of analyzing such data has led to the birth of a unifying framework, data science with graphs. This framework offers a mathematically rigorous paradigm to analyze high-dimensional data associated with complex and irregular graph structures. It extends tools and concepts from classical signal processing and machine learning to the graph domain. In this talk, we will introduce this emerging framework and consider a series of graph-based theories and methodologies, including sampling theory of graph data and graph-based deep neural networks. We will illustrate the power of the proposed methodologies with the applications to social network analysis, urban spatial-temporal data analysis, and autonomous driving. > > Bio: Siheng Chen is a research scientist at Mitsubishi Electric Research Laboratories. Before that, he was an autonomy engineer at Uber Advanced Technologies Group, working on the perception and prediction systems of self-driving cars. Before joining Uber, Dr. Chen was a postdoctoral research associate at Carnegie Mellon University. Dr. Chen received his doctorate in Electrical and Computer Engineering from Carnegie Mellon University in 2016, where he also received two masters degrees in Electrical and Computer Engineering (College of Engineering) and Machine Learning (School of Computer Science), respectively. He received his bachelor’s degree in Electronics Engineering in 2011 from Beijing Institute of Technology, China. He is the recipient of the 2018 IEEE Signal Processing Society Young Author Best Paper Award. His coauthored paper received the Best Student Paper Award at 2018 IEEE Global Conference on Signal and Information Processing. His research interests include signal processing, machine learning, 3D point cloud processing, and autonomous driving. > > Best, > > Jia-Bin Huang > https://filebox.ece.vt.edu/~jbhuang/ > > -- > You received this message because you are subscribed to the Google Groups "ECE faculty (ece-faculty-g)" group. > To unsubscribe from this group and stop receiving emails from it, send an email to ece-faculty-g+unsubscribe(a)vt.edu. > To post to this group, send email to ece-faculty-g(a)vt.edu. > _______________________________________________ > Dac-faculty mailing list > Dac-faculty(a)cs.vt.edu

1 0

Wednesday, March 20th @ 4 PM, McB 133C, Xinfeng Xu
by Alexander Rodriguez 18 Mar '19

18 Mar '19

Please join us for our next DM-Meeting. ****DM-Meeting Spring 2019**** WHO: Xinfeng Xu WHEN: Wednesday, March 20th @ 4 PM WHERE: McB 133C WHAT: Xinfeng will practice his master thesis defense "" Title: Modeling and Predicting Incidence: Critical Systems Failures and Flu Infection Cases Abstract: Given several related critical infrastructure (CI) networks, such as power grid, transportation, and water systems, one crucial question emerges: how to model the propagation of failed facilities and predict their spread over time to the whole system? Given digital surveillance data, can we predict the impact of Influenza-Like Illness (ILI), including the percentage of outpatient doctor visits, the season duration, and peak? These questions are related to modeling and predicting the incidence of different types of contagions. In the case of CI, the contagions are the failures of facilities. In the case of flu spread, the contagions are the infective ILIs. In this thesis, in the case of CI, we give a novel model of failure cascades and use it to identify key facilities in an optimization-based approach, called HotSpots. In the case of flu spread, we develop a deep neural network, EpiDeep, to predict multiple key epidemiology metrics. In both of these applications, we use the dynamics of propagation to develop better algorithms.\\ By collaborating with Oak Ridge National Laboratory (ORNL) and working on the real CI networks provided by them, we find that HotSpots helps solve what-if scenarios. By carrying on experiments on the digital surveillance data from the Centers for Disease Control and Prevention (CDC), we find that EpiDeep is better than non-trivial baselines and outperforms them by up to 40\%. We believe that our results are general and can be applied to other propagation-based scenarios in infrastructure and epidemiology. Best regards, Alex _______________________________________________ Seminar website: *http://people.cs.vt.edu/~arodriguez/DM-Meeting/home.html <http://people.cs.vt.edu/~arodriguez/DM-Meeting/home.html>* Please contact Alexander Rodriguez (arodriguez(a)cs.vt.edu <esorour(a)cs.vt.edu>), if interested in giving a talk. Dm-meeting mailing list Dm-meeting(a)cs.vt.edu https://mailman.cs.vt.edu/mailman/listinfo/dm-meeting

1 0

Fwd: [Bburg-fac] Invitation to final defense of Ph.D. candidate Yufeng Ma, soon to go to Yahoo!
by B. Aditya Prakash 15 Feb '19

15 Feb '19

FYI! -------- Forwarded Message -------- Subject: [Bburg-fac] Invitation to final defense of Ph.D. candidate Yufeng Ma, soon to go to Yahoo! Date: Fri, 15 Feb 2019 10:41:14 -0500 From: EdFox via Bburg-fac <bburg-fac(a)cs.vt.edu> Reply-To: EdFox <fox(a)vt.edu> To: CS Faculty <faculty(a)cs.vt.edu> All are invited to the final defense of Ph.D. candidate Yufeng Ma, soon to go to Yahoo! *Time*: 2:30-4:30 PM EST, 02/20 (Wednesday) *Location*: 2030E Torgersen Hall (DLRL) *Title:* Going Deeper with Images and Natural Language *Abstract:* One aim in the area of artificial intelligence (AI) is to develop a smart agent with high intelligence that is able to perceive and understand the complex visual environment around us. More ambitiously, it should be able to interact with us about its surroundings in natural languages. Thanks to the progress made in deep learning, we've seen huge breakthroughs towards this goal. The developments have been extremely rapid in visual recognition, in which machines now can categorize images into multiple classes, and detect various objects within an image, with the ability that is competitive with or even surpasses that of humans. Meanwhile, we also have witnessed similar strides in natural language processing (NLP). It is quite often for us to see that now computers are able to almost perfectly do text classification, machine translation, etc. However, despite much inspiring progress, most of the achievements made are still within one domain, not handling inter-domain situations. The interaction between the visual and textual areas is still quite limited, although there has been progress in image captioning, visual question answering, etc. In this dissertation, we design models and algorithms that enable us to build in-depth connections between images and natural language, which help us to better understand their inner structures. In particular, first we augment the conventional image captioning model using adversarial loss, to boost its performance on various metrics (e.g., BLEU, CIDEr). Generative adversarial networks (GANs) are connected with discrete caption data through the Gumbel-Softmax trick. The results of our experiments with the MSCOCO caption dataset show that our model is capable of generating captions with improved quality and higher readability. Second, we develop a model (Quasi-Supervised Learning network) for measuring review congruence, which takes an image and the review text as input and quantifies the relevance of each sentence to the image. The whole model is trained in a purely unsupervised way, much like GANs. Although there are no manual tags involved in training the model, we build pseudo labels with the help of attention mechanisms. The model is able to accurately estimate the relevance of sentences to an image based on our experimental results on the Yelp Restaurant dataset. Lastly, we go deeper with fine-grained image understanding by identifying aspect level ratings and by detecting corresponding aspects from the image within the review context, which is considered a novel AI task. This would help merchants provide better service and readers make choices more easily. Adaptive models are proposed to automatically adjust weights placed on images and texts for overall rating prediction. It shows significant improvement as compared to our baseline model based on the Mean Squared Error (MSE) metric in terms of both overall and aspect level rating. On the theoretical side, this research contributes to multiple research areas in Computer Vision (CV), Natural Language Processing (NLP), interactions between CV&NLP, and Deep Learning. Regarding impact, these techniques will benefit related

1 0