- Migration - mailman.cs.vt.edu

VT Migration Prediction Update
by Brian Mayer 17 Mar '23

17 Mar '23

Alex, Here is the final biweekly update. I am only including you on the distribution. Feel free to send it broader. Holly will receive a copy with the invoice as requested. Feel free to give me a call next week and we can discuss. Thanks, Brian -- Brian Mayer bmayer(a)cs.vt.edu 540-231-5907 Sanghani Center for Artificial Intelligence and Data Analytics <https://sanghani.cs.vt.edu/> - Virginia Tech <https://vt.edu/>

1 0

Final Report
by Brian Mayer 06 Mar '23

06 Mar '23

Alex, Here is our Final Report and Lessons Learned as discussed during our call on Tuesday. Let me know if there is anything else you'd like us to add. Also as mentioned over the next two weeks we are working to prepare the sample product output and all the elements that are going into it. Happy to set up another call if you'd like and let us know if/when you have dates for an in-person meeting. Thanks, Brian and team -- Brian Mayer bmayer(a)cs.vt.edu 540-231-5907 Sanghani Center for Artificial Intelligence and Data Analytics <https://sanghani.cs.vt.edu/> - Virginia Tech <https://vt.edu/>

2 1

VT Migration Product Output Outline
by Brian Mayer 02 Mar '23

02 Mar '23

Alex, I've attached an outline of what we think the output could include. I've discussed these items with the team and we are confident most of them will come to fruition in the next two weeks (or sooner) but a few are still being tweaked, tested, or completed so we may not be able to include them in the final product for this effort. I have also listed a few options of items on page 3 that could further enhance the report but are items that we definitely won't be able to get to in the next two weeks but could certainly work on if we have more time. One of those is a list of the fastest-growing terms, that I know you seemed to be interested in. I'll be working tonight and tomorrow on converting our slides into a document and attaching this sample output as an Appendix for the final report (which I will send to you tomorrow). Thanks, Brian -- Brian Mayer bmayer(a)cs.vt.edu 540-231-5907 Sanghani Center for Artificial Intelligence and Data Analytics <https://sanghani.cs.vt.edu/> - Virginia Tech <https://vt.edu/>

2 1

Notes from my discussion with Alex
by Brian Mayer 28 Feb '23

28 Feb '23

In addition to any quantitative predictions we provide (total, by sector, and by country) in the current dashboard... Alex would like us to provide what demographics we expect to see an uptick from. (*Bursts*?) - We could look for bursts of keywords by demographic - country - family type - sector - conveyance (????) - We know that this is pretty reliable as a total and by sector, but we may need to validate others. Might be wise to develop a more objective validation methodology - e.g., What defines an uptick/burst in the encounter data (increase by X%?) - we tried using burst analysis but for some reason this didn't work - Then maybe we determine Accuracy/Precision/Recall - We will only be able to evaluate a limited amount of observations (20-30) - SHOULD WE DO BURST ANALYSIS? HOW MUCH? HOW SHOULD WE VALIDATE? Alex liked the fastest-growing keywords from the EMBERS visualization (CAN WE PROVIDE THIS? CAN WE INCLUDE NON-KEYWORDS? RUN DQE?) Alex liked the word cloud in the EMBERS visualization (WHAT COULD/SHOULD WE USE TO CREATE A WORD CLOUD?) Alex liked the idea of "hot" keywords Other aspects that could be included with "Hot" Keywords - Group hot keywords by spaCy entity type, e.g., PERSON, NORP, FAC, ORG, GPE, etc. - Can we include non-keywords *Modeling work still to do in addition to anything determined above* Take another look at the country-specific predictions and try to improve nMIL results Confidence Interval CBP Conveyance/Transportation analysis (*push this to future work*) *Items Due to ODNI* Provide Alex an outline of the product Turn slides into document report Add a description/example of the output product to the final report -- Brian Mayer bmayer(a)cs.vt.edu 540-231-5907 Sanghani Center for Artificial Intelligence and Data Analytics <https://sanghani.cs.vt.edu/> - Virginia Tech <https://vt.edu/>

1 0

Contract status update: 2022-22083100006 Migration (21FEB2023)
by Holly T. Phelps 21 Feb '23

21 Feb '23

UNCLASSIFIED Good afternoon Brian and team - I'm reaching out to provide a level-set on the migration project, as I understand it from the COTR perspective. You may recall the initial statement of work had a POP that ended 24 February (this coming Friday); the finalized contract lists the period of performance as ending 18 March 2023. I am working to correct the dates in the SOW, but the update may not be processed this week. The date is meaningful as I understand that you are working on the "final report on lessons learned", which is due no later than 165 days from the contract start date. Counting from 19 September (the date in the contract), 165 days would be 3 March 2023. The team look forward to receiving your report. To date, I have received and approved two invoices for this contract totaling $250,000; however, no paper documentation accompanied the request in the system or by email. If this was in error, and there were written documents meant to accompany the invoice in IPP, please share them so that I can file them appropriately. One administrative ask: could you please amend your distro to include this email (@odni.gov) and the cc'd moving forward? ODNI is transitioning mail systems, and while I receive mail at the ucia.gov account, I no longer have direct access to the mailbox. I'd appreciate your help in making sure I can keep the contract file for this project complete! Please reach out if you have any end-of-contract questions. V/r, Holly Holly Phelps holly.t.phelps(a)odni.gov (301) 243-0658 UNCLASSIFIED

1 0

VT Migration Prediction Update
by Brian Mayer 17 Feb '23

17 Feb '23

All, Please find attached our biweekly status report. Thanks, Brian and team -- Brian Mayer bmayer(a)cs.vt.edu 540-231-5907 Sanghani Center for Artificial Intelligence and Data Analytics <https://sanghani.cs.vt.edu/> - Virginia Tech <https://vt.edu/>

1 0

Re: [Migration] Product Output/Visualizations
by Brian Mayer 17 Feb '23

17 Feb '23

Thanks for this Alex. I was already starting to think about the suggestions you made in your last email but this has more detail that we can work on further. If we don't get something set up with NIM and D/NIM in the next two weeks we should plan a meeting with you before then, even if just by Zoom, to discuss this output but give us a week to brainstorm and come up with some responses. 12, 2 or 4 are good times for me tomorrow. I am booked until 12, from 1-2, and sort of have something around 3. On Thu, Feb 16, 2023 at 8:15 PM Flynn A Rapavi <FLYNNAR(a)dni.gov> wrote: > Brian, > > > > I will try and give you a call tomorrow or next week to discuss some > product output ideas in detail. > > > > In the meantime, and in general, I think a baseline text-annotated > visualization product that offers the following elements is what the NIM > and D/NIM (and DMI front office) and even the DNI are seeking—essentially, > a forecast paper (one or two pages), that discusses and depicts > graphically, projected, possible migrant departure points (from whichever > LATAM country/countries) – perhaps a graphic showing departure points > plotted to a map, potential points of arrival (e.g. what CBP sectors along > the SWB with estimated numbers), perhaps a visual showing potential migrant > routes from points of departure to points of entry or similar, anything on > methods of migrant transit/transport, total migration number estimates > (monthly or eventually shorter turn durations at or greater than 100 > persons per parameters), along with country/citizenship and demographic > data (like the data characterized in the histograms shown in the monthly > reports). The output paper might have an overarching analytic statement > from the team regarding the prediction and some form of confidence > statement/maybe in the form of a percentage. Comparative trends on overall > accuracy of predictions e.g. showing improved correlation/covariance might > be rolled into the summary from the next product iteration – in other > words, to keep showcasing at a high-level how model accuracy is improving > over time. > > > > This product would represent the output most likely shared with and used > by senior leaders and key partners. Leadership is seeking higher-level > products (as compared to what I’ve been discussing with them based on the > contents of the monthly reports). We (at least me) definitely still need > all of the back-end specifics that go into that predictive paper like the > data you/team have been providing right along – > > > > *All that said, I wanted to provide at least a rough idea of what it is I > truly think leadership here is looking for in terms of MRP output*. I > assume this is what the NIM and D/NIM will want to see at our future > meeting, as well. > > > > Additionally, I am also working on next steps. I have a meeting with the > CO/COTR tomorrow and want to ascertain at least to some degree what we can > hope to do, moving forward. I hope to be able to have a discussion on that > soon – and definitely by our next meeting. > > > > Finally, Jim and I am still trying to lock down a meeting time but > schedules and OPTEMPO are making things problematic. We will keep after it. > > > > I again apologize for the delay in responding – but I do hope this helps. > > > > Always best regards, > > Alex > > > > > > [image: Office of the Director of National Intelligence Seal] > > > *Alexander Rapavi* > > Program Mission Manager > > GTOC/IN/CN > > NIM-WHT > > Office of the Director of National Intelligence > > 703.275.2711 > > > > > -- Brian Mayer bmayer(a)cs.vt.edu 540-231-5907 Sanghani Center for Artificial Intelligence and Data Analytics <https://sanghani.cs.vt.edu/> - Virginia Tech <https://vt.edu/>

2 1

Re: [Migration] FOLLOW-UP MEETING: REQUEST (Possible NIM-WHT, D/NIM WHT & PMM GTOC visit to VA TECH Sanghani Center on Wednesday 8 February 2023 from 2:00-3:30: VT Migration Midpoint Report on Lessons Learned)
by Brian Mayer 06 Feb '23

06 Feb '23

Alex, We have reserved the room again for Wednesday from 2-330. I will send a meeting invite to you and James. We were planning to mostly walk through the slides we presented last week. We will also add slides for new work/results since then. Were there any additional specific items you wanted us to discuss/cover? Thanks, Brian On Mon, Jan 30, 2023 at 6:03 PM Flynn A Rapavi <FLYNNAR(a)dni.gov> wrote: > Naren, > > > > This is great news. > > > > I will meet with both Jeanette and Nicole again and will work to help them > develop a list of things they would like to see/topics to cover – some of > the requests for specifics might include those coming from our > organization’s top-level FO (MRP is decidedly getting a lot of attention to > include from the DNI). > > > > This session should be very beneficial, I think- > > > > I am certainly looking forward to it- > > > > Best regards, > > Alex > > > > [image: Office of the Director of National Intelligence Seal] > > > *Alexander Rapavi* > > PMM, GTOC/IN/CN > > NIM-WHT > > Office of the Director of National Intelligence > > 703.275.2711 > > > > > > > > > > *From:* Naren Ramakrishnan <naren(a)cs.vt.edu> > *Sent:* Monday, January 30, 2023 5:36 PM > *To:* Flynn A Rapavi <FLYNNAR(a)dni.gov> > *Cc:* Brian Mayer <bmayer(a)cs.vt.edu>; James M. Robinson <JAMESMR6(a)dni.gov>; > Juanita Victoria <juaniv6(a)vt.edu>; migration(a)cs.vt.edu > *Subject:* Re: FOLLOW-UP MEETING: REQUEST (Possible NIM-WHT, D/NIM WHT & > PMM GTOC visit to VA TECH Sanghani Center on Wednesday 8 February 2023 from > 2:00-3:30: VT Migration Midpoint Report on Lessons Learned) > > > > Hi Alex > > > > Definitely, happy to support a visit with the NIM and DNIM at the proposed > date and time. > > > > Let us know if there are additional analytic aspects we should be working > on toward this visit. > > > > Best > > Naren > > > > On Mon, Jan 30, 2023 at 4:56 PM Flynn A Rapavi <FLYNNAR(a)dni.gov> wrote: > > Brian, > > > > Sincere thanks to Naren, you and the team for hosting us last Friday. > Supremely beneficial from my vantage point. Also decidedly appreciate the > materials you forwarded (100% needed) – > > > > I have socialized the briefing and other related information with the > National Intelligence Manager for Western Hemisphere and Transnational > Crime (Jeanette Rivera-Breznai) and Deputy National Intelligence > Manager-WHT, Nicole Peruyera— > > > > Both were very receptive and eager to see additional output and forecast > analyses, as well as discuss the possible development of specific tailored > analytic products and next steps. > > > > *Our Friday visit was so successful, NIM-WHT is hopeful Naren, you and the > team can support a visit on 8 Feb 2023 from 2:00-3:30 –* > > > > *Prospective Attendees:* > > > > - *Jeanette Rivera-Breznai (NIM-WHT)* > > - *Nicole Peruyera (D/NIM-WHT)* > > - *Alexander Rapavi (PMM, GTOC/CN)* > > > > (U) I have included James (Jim) M. Robinson on the cc line. Jim is > NIM-WHT’s Exec and executes all the important things that happen around > here- > > > > (U) Sincere thanks in advance for your consideration and assistance > (again). Hope we can make this happen. > > > > Best regards, > > Alex > > > > [image: Office of the Director of National Intelligence Seal] > > > *Alexander Rapavi* > > PMM, GTOC/IN/CN > > NIM-WHT > > Office of the Director of National Intelligence > > 703.275.2711 > > > > > > > > > > *From:* Brian Mayer <bmayer(a)cs.vt.edu> > *Sent:* Friday, January 27, 2023 12:03 PM > *To:* Flynn A Rapavi <FLYNNAR(a)dni.gov> > *Cc:* migration(a)cs.vt.edu > *Subject:* VT Migration Midpoint Report on Lessons Learned > > > > Alex, > > Please find attached our slides from today. > > > > I did add one slide that lists the Top 60 keywords we are currently using > in case you have feedback on that. > > > > Brian > > > > -- > > Brian Mayer > > bmayer(a)cs.vt.edu > > 540-231-5907 > > Sanghani Center for Artificial Intelligence and Data Analytics > <https://sanghani.cs.vt.edu/> - Virginia Tech <https://vt.edu/> > > -- > > Naren Ramakrishnan <http://people.cs.vt.edu/~naren/> > > Thomas L. Phillips Professor, CS@VT <https://www.cs.vt.edu> > > Director, Sanghani Center for AI and Analytics > <https://sanghani.cs.vt.edu> > > Director, Amazon-VT Initiative in Efficient & Robust ML > > Director, NSF NRT UrbComp > <https://sanghani.cs.vt.edu/academics/urban-computing/> > > On Twitter as @profnaren <https://twitter.com/profnaren> > > > > > > -- Brian Mayer bmayer(a)cs.vt.edu 540-231-5907 Sanghani Center for Artificial Intelligence and Data Analytics <https://sanghani.cs.vt.edu/> - Virginia Tech <https://vt.edu/>

1 1

VT Migration Prediction Update
by Brian Mayer 03 Feb '23

03 Feb '23

All, Please find attached our biweekly status report. Thanks, Brian and team

1 0

Re: [Migration] FOLLOW-UP MEETING: REQUEST (Possible NIM-WHT, D/NIM WHT & PMM GTOC visit to VA TECH Sanghani Center on Wednesday 8 February 2023 from 2:00-3:30: VT Migration Midpoint Report on Lessons Learned)
by Naren Ramakrishnan 30 Jan '23

30 Jan '23

Hi Alex Definitely, happy to support a visit with the NIM and DNIM at the proposed date and time. Let us know if there are additional analytic aspects we should be working on toward this visit. Best Naren On Mon, Jan 30, 2023 at 4:56 PM Flynn A Rapavi <FLYNNAR(a)dni.gov> wrote: > Brian, > > > > Sincere thanks to Naren, you and the team for hosting us last Friday. > Supremely beneficial from my vantage point. Also decidedly appreciate the > materials you forwarded (100% needed) – > > > > I have socialized the briefing and other related information with the > National Intelligence Manager for Western Hemisphere and Transnational > Crime (Jeanette Rivera-Breznai) and Deputy National Intelligence > Manager-WHT, Nicole Peruyera— > > > > Both were very receptive and eager to see additional output and forecast > analyses, as well as discuss the possible development of specific tailored > analytic products and next steps. > > > > *Our Friday visit was so successful, NIM-WHT is hopeful Naren, you and the > team can support a visit on 8 Feb 2023 from 2:00-3:30 –* > > > > *Prospective Attendees:* > > > > - *Jeanette Rivera-Breznai (NIM-WHT)* > > - *Nicole Peruyera (D/NIM-WHT)* > > - *Alexander Rapavi (PMM, GTOC/CN)* > > > > (U) I have included James (Jim) M. Robinson on the cc line. Jim is > NIM-WHT’s Exec and executes all the important things that happen around > here- > > > > (U) Sincere thanks in advance for your consideration and assistance > (again). Hope we can make this happen. > > > > Best regards, > > Alex > > > > [image: Office of the Director of National Intelligence Seal] > > > *Alexander Rapavi* > > PMM, GTOC/IN/CN > > NIM-WHT > > Office of the Director of National Intelligence > > 703.275.2711 > > > > > > > > > > *From:* Brian Mayer <bmayer(a)cs.vt.edu> > *Sent:* Friday, January 27, 2023 12:03 PM > *To:* Flynn A Rapavi <FLYNNAR(a)dni.gov> > *Cc:* migration(a)cs.vt.edu > *Subject:* VT Migration Midpoint Report on Lessons Learned > > > > Alex, > > Please find attached our slides from today. > > > > I did add one slide that lists the Top 60 keywords we are currently using > in case you have feedback on that. > > > > Brian > > > > -- > > Brian Mayer > > bmayer(a)cs.vt.edu > > 540-231-5907 > > Sanghani Center for Artificial Intelligence and Data Analytics > <https://sanghani.cs.vt.edu/> - Virginia Tech <https://vt.edu/> > -- Naren Ramakrishnan <http://people.cs.vt.edu/~naren/> Thomas L. Phillips Professor, CS@VT <https://www.cs.vt.edu> Director, Sanghani Center for AI and Analytics <https://sanghani.cs.vt.edu> Director, Amazon-VT Initiative in Efficient & Robust ML Director, NSF NRT UrbComp <https://sanghani.cs.vt.edu/academics/urban-computing/> On Twitter as @profnaren <https://twitter.com/profnaren>

2 1