DeepSensor conversation

--- tags: deepsensor --- # DeepSensor conversation ## Notes - Relationship between IceNet and DeepSensor - Tom led the IceNet research initially, and then funding ran out, and then he was moved on to a new project, and then he turned that into DeepSensor. - Sparse connections: One of DeepSensor's key/novel capabilities is that it can handle multi-resolution data and different kinds of environmental data modalities—like station data and gridded data. IceNet's goal is just to forecast sea ice as well as you can across the polar oceans, and it doesn't really help much, in Tom's opinion, to have station inputs. = The technology behind DeepSensor won't be that helpful to IceNet. - Different problem spaces but could link up one day (not on the current agenda) - On the agenda: integrating the outputs of IceNet with the outputs of DeepSensor for proposing sensor trajectories. (That's a plan for AI UK 2024.) So: Connection between IceNet and DeepSensor = through the outputs. - Part of a very broad ASG program, attempting to combine satellite data with point-wise station data, essentially trying to tackle this challenge of blending different kinds of environmental data, intelligent fusion of data, intelligent monitoring of environmental changes. It was Scott's brainchild, and then Tom steered his part of it to the direction of sensor placement and using neural processors, and that eventually turned into this idea of having a Python package to do those things—that became DeepSensor. - Future of DeepSensor - Scott has written it into the E&S GC as a deliverable and sub-component. - There are some ambitions to move on to sensor trajectories for autonomous underwater vehicles, etc. But otherwise, it's just support for DeepSensor in a general sense. - One of Tom's key research interests going forward is to look at infilling missing satellite data, due to cloud coverage for example. Building that into DeepSensor would mean other people can use it. - REG people have expressed interest in being involved = a force multiplier application. Extensibility would be helpful. - Getting to version 1.0 - Submission to JOSS - Engagement from users/contributors - DeepSensor, like the concepts behind it, is quite complex. Tom has tried to implement the package in a simple way, but thinks contributors and developers will benefit from initially just being users and playing around with it, watching some technical talks, skimming some papers, because the ML aspect is more nuanced and complex than a standard multi-layer perceptron or CNN type thing. - Tom is interested in getting more regular contributors who are autonomous, to build up a community and an online space and repository where it’s amenable to having that possibility—provide people with the right resources to learn from, in order to get to become contributors. - There has been initial conversations with Scivision but nothing tangible yet—there may be a connection in DeepSensor's potential to help with satellite data processing. - Potential user community currently consists predominantly of the Cambridge group: - Wessel Bruinsma - Anna Vaughan - [Stratus Marco] - [James Requimer] - Jonas Scholz - Also some contributors to the GitHub repository: - Zeel Patel - Rohit Singh Rathaur ## Transcript **Alden is great and has been such a resource in terms of explaining absolutely everything. For me, just what's still sort of moving around in my brain is a little bit like the relationship between all of these different things [IceNet and DeepSensor]. Um, so just getting that sort of straightened out, I think, would be super helpful for me.** Yeah, well, it's quite simple because there isn't really a relationship. So, that's the kind of simplest answer. The most obvious relationship is that I kind of led the IceNet research initially, and then funding ran out, and then I was moved on to the new project, and then I, you know, turned that into DeepSensor. So, it's like, you know, I am the commonality across the projects. In terms of their conceptual connections, it's pretty sparse because, um, one of DeepSensor's key capabilities that is novel is that it can handle multi-resolution data and different kinds of environmental data modalities like station data and gridded data. Whereas IceNet's goal is just to forecast sea ice as well as you can across the polar oceans, and it doesn't really help that much to have station inputs, in my opinion. I haven't verified that. I just know IceNet predicts on a fairly coarse resolution; at the moment, it's 25 kilometers, and we're trying to improve that. I just don't think point-wise stations will help, so the technology behind DeepSensor won't be that helpful. **So they're sort of addressing different problems really or different problem spaces?** But they could link up one day. It's just not really on the agenda. And, um, one thing we want to do is integrate the outputs of IceNet with the outputs of DeepSensor for proposing sensor trajectories. That's kind of what our AI UK 2024 plan is, or one of them. When we talk about them at the same time as if they're connected, that's what we mean—outputs. **That's helpful to know, especially because I know that Scott has this sort of grandiose plan for AI UK 2024. Now, I'm also on the planning committee for that, and so, like, I feel like he is as well. So, there's everything lining up to make it into something really great. So it's good to know that that's on the horizon for everyone involved. I think that's a good movement forward. But so, you said DeepSensor followed after your work on IceNet. Is that the timeline of IceNet? So how did DeepSensor come about then? Or was that something that existed previous to you, or is it your brainchild, the whole thing?** No, Scott's brainchild. And let me just see if I can bring up, um, the project webpage, the Turing, the old initial Turing project webpage. I'm just going to put it in chat once I confirm that it actually loads on my laptop. So, essentially, it was very broad. It was just about kind of combining satellite data with point-wise station data, essentially trying to tackle this challenge. It's in chat, this challenge of blending different kinds of environmental data, intelligent fusion of data, intelligent monitoring of environmental changes. So it's very broad. It was Scott's brainchild, and then, you know, I steered my part of it down the direction of sensor placement and using neural processors, and that eventually turned into this idea of having a Python package to do those things, that's DeepSensor. **So is that part of this? Because this is more than just DeepSensor. The environmental monitoring is more of an overarching... This has been a struggle for me coming into the Turing, by the way. Just the website is like there are 15 million pages that you never knew even existed because they're buried four levels down. But yeah, I get a sense that this is obviously some kind of overarching part so that DeepSensor sits in this as part of it.** Well, this is, yeah, this is finished. It ended with ASG, but... DeepSensor came out of Work Package 1 and there were three work packages. So essentially, Scott's written it in as one of the deliverables and sub-components of this Grand Challenge. So, DeepSensor is living on, and there are some ambitions to move on to sensor trajectories for autonomous underwater vehicles and things. But otherwise, it's just support for DeepSensor in a general sense. One of my key research interests going forward is to look at infilling missing data in satellites, like, for example, due to cloud coverage, right? So that's something that I'm going to hopefully have time to focus on and build into DeepSensor so other people can use it. And then we are having REG people, hopefully, come on board. Some REG people have expressed interest in being involved in a kind of force multiplier project, a so-called force multiplier project, which is essentially that REG is going to have some spare RSEs -- not spare, but just RSEs. They're never spare, to be like focusing just on the Turing kind of main challenges and to focus on projects where a bit of concerted effort can have this scale-multiplying effect. And the idea is that DeepSensor would be a candidate for that. Because by having this open-source software, if we make it more robust and build a community, then that can expand out and improve a lot of research. **That's the application that we needed to get in by Friday, but for the other thing that I was working on, it's a force multiplier REG resource. So, we're all in that process. Have you gotten your application approved? And do you have REG members then that are going to be part of it, or no?** Not confirmed. I did check in with Oliver because I hadn't had the update in a while, and he told me what I now connect with what you're saying. He says they're having a meeting on Friday for all the force multipliers, and a few people have expressed interest, and I'll have an update on Friday. **From what I know from that process, now considering that I have dug extremely deep into that for the past 24 hours, it seems like as long as there's interest and there are REG members, like you're saying they're "spares", they have time. And the whole point of it is right, there's this time period now between the end of ASG and the beginning of the grand challenges that we can dip into. And it seems like, I mean, for me, I don't know if I'm being overly optimistic, but that was the sense I got from talking about my application. As long as you get a green light from someone senior in REG about it, Oliver being one (and in my case, I was talking to Dave), you kind of get this sense that it's really about just motivating your project and showing that you have something lined up. And it sounds like, I mean, I was obviously following along with your talk the other week, so I have a good understanding.** Yes, very closely. **As close as I could, as close as I followed that time. So it was like, "Well, he lost me." Not your fault, but just that sort of going, and I think you did a great job of prefacing with, "Here we go, some of you will get lost, and it's fine. I'll get back to you."** Yeah. **It's a sort of optional listening, but I think I have a good sense of how it works and what sort of problems it addresses and all that stuff. But it's good to know a little bit more, I guess, of where you're seeing it going. And the force multiplier then would be sort of a first step, I guess, in what comes next. Like, what are the next steps? Because you mentioned already sort of the directions in which you wanted it to go in the talk, and I think that would be something that would be interesting to talk a little bit more about. I'm bringing my notes up, and I can't find them now.** Yeah, no, it's fine. Now, what you talked about, like scaling it up and you talked about applying transfer learning techniques... Yeah. **...integrating with other sort of architectures, maybe beyond PyTorch, I guess? More like...** Yeah, extensibility of the model. So like, is it... I've tried to design it to be extensible, but is it actually possible for someone to come and add that new model and have everything just work? **Yeah, yeah. So, is that the idea for the force multiplier then? To kind of have that as a testing ground or a playground for using DeepSensor more so than developing it, maybe?** It's one idea. I do think that we can touch on this, but DeepSensor, like the concepts behind it, are quite complex. I've tried to implement it in a simple way, but I think contributors and developers will benefit from initially just being a user and playing around with it, you know, watching some technical talks, skimming some papers, because the ML aspect is more nuanced and complex than your standard multi-layer perceptron or CNN type thing. **Right.** So, like, you know, suddenly there are tasks with all these different sets of observations and things, rather than just like a single batch or something where there's a tensor in and a tensor out or something. But yeah, extensibility would be, I think, high on the list of things where the RSE could be helpful. **Right. That's good to know. So, but right now, you're the only person? Because I looked at the contributors to the code base, and it looks like you're the sole contributor, more or less. I mean, there are other people who have contributed little bits and pieces, but it was like 24,000 lines of the code was written by you, and then like 100 here, 100 there by other people or whatever.** Wow, is that 24? Oh my God, true. Okay, so that's mainly groups of notebooks. Yeah, it's good to find a way to get rid of that because like Jupyter notebooks, in their raw form, are like huge. **Exactly. Like some kind of nice-looking JSON format that's just extremely long and tedious. Nice big numbers on the contribution though.** So, I guess that's true. But correct, like, I'm the main developer and contributor. Yeah, and yeah, that. I've been very lucky to have some other contributors, but it has been quite minor, their contributions so far, which is fairly to be expected given the early stages of this package. But one thing that I would like to discuss with you is how we get more regular contributors who are kind of autonomous, not completely necessarily, but just build up a community and an online space and repository where it's amenable to having that be possible. You know, people have the right resources to learn from to get up to that stage, etc. **Yeah, yeah. So one of the things that I just sort of looking through the repository and everything, like, you have a lot of things in there, but there's a lot of foundational contributor stuff, like how to contribute, right? That would be a good sort of community management thing. You know, The Turing Way has all of that modeled out, and it's really easy for us to think about the next steps with that. And just documentation. I've seen that you have the docstrings in there, but there's no way for it right now to sort of render out. But that is also something that I have done in Sphinx and Read the Docs in previous projects quite a bit, just improving documentation and then also generating the public-facing side of that. But then I also see the potential of something like this tying in with the Scivision community that is already existing. I don't know to what extent you've been in touch with that. I mean, obviously through Alden, I assume that those lines have been crossed.** Yeah, yeah. Conversations have been exchanged, but nothing super tangible. I think that is a connection because DeepSensor has the potential to help with satellite data processing. **Yeah, and I know I don't know too much about Scivision, to be fair either. But I was working on MapReader, which was the thing I think that led to Scivision initially. I'm not 100% sure about the timelines there, but I was on the MapReader project, and that was obviously applying computer vision to historical maps, which then became applied in biology and people using the same techniques to look at leaf formations or something like that, which then expanded into becoming this whole other package that is more about computer vision generally. But in those conversations that we had around MapReader, a lot of people were like, 'But we need to apply this to satellite imagery.' So, I think that there is a lot of potential users. That's obviously slightly different, maybe a different community, but maybe there could be conversations to be had around what Scivision actually means. Like, this could be a way to think about expanding the sense of vision there, and what does it mean in terms of tying in with an existing community or trying to expand that definition. Because another way of going, of course, would be to build a specific community around DeepSensor. But in my experience, it's always good to rely on already existing infrastructure, at least to start building things because you already have the people that have the knowledge and the potential technological competency to become contributors and users of a thing like this. So anyway, that was my initial thought, just from looking through the code base very briefly. And I saw also that you have the DeepSensor gallery, which I haven't looked at too much, but obviously there's lots to test out there in terms of content and examples. So, I don't know, to some extent, I'm like, you've already done all that work.** I think maybe improved a lot. Yeah, yeah. It's very preliminary. It's like I'm just putting this here as a kind of reminder, like a post-it note that's literally in the repository. It's kind of like, This will do for now because I don't have time to do more, but I want this to be improved." **Yeah, yeah. And at some point, have someone come back to it, and potentially that's exactly the role that someone like I could have, to actually just make sure that we circle back to those things and make sure they get taken up and dealt with. Do you know if there are other people using DeepSensor yet, or if there are other use cases beyond our own people, instead of the ones that we already know?** Yeah. Okay, so there's the people we already know, and I don't know if you know everyone there.** **I might not know everyone there, but so that might be a good thing to also loop me in with who they are. But yes, go ahead.** So, I guess the research behind DeepSensor primarily came out of a group at Cambridge that I've been collaborating with, led by Professor Rich Turner. And right, his posse of PhD students and ex-PhD students, who have worked on neural processes and focused on environmental applications and things. For example, [Vessel Grinsma], Anna Vaughn, [Stratus Marco], [James Requimer]. Then, Rich and I had a master's student, Jonas Schultz, who I mentioned for one side of the talk. He has been researching Sim2real using Deep Sense, probably one of the main users so far. He's made some contributions and provided useful feedback. The others haven't used it yet, but maybe they would be interested one day. One issue of DeepSensor is that it focuses more on being super simple and easy to use. That means if you want to customize things heavily, it might not be flexible enough. But, you know, if there's some really specific niche thing that you need that's missing... But then, that's quite abstract. I'm not giving you a tangible example of something that's missing. But it'd be good to kind of hear from people. Like, have a look at this. Would this suit your research? Would there be something missing, or something you'd want to do? **Yeah, that ties in with what you're saying about the extensibility then. And testing that, making sure that people can try to apply it and see if it works. And if it doesn't, then go back to opening an issue, or like trying to solve something in the code base itself, I guess.** Yeah, it definitely touches on that. And just like, the way the data handling is. Is that flexible enough? The task loader class that I have is very flexible, but it might not do everything. And the forecasting functionality... there's several people interested in forecasting. Like Anna Vaughan. She does her research in her own code. She doesn't even use Vessel's really nice neural processes package. She writes her own models just because she likes that kind of level of control. But she would be interested in having her code written using DeepSensor because then it's way easier. For example, if someone emails her asking 'where's your code?', she can pass them on to a much more extensible and usable code base. **Yeah.** And so I'm just kind of trying to remember everything and dump all the information, knowing that it's recorded. **Yeah, exactly. No, for sure. I mean, it's great for me to just have this and be able to dig into it and potentially follow up with you and talk about all of these different things. That's great.** That's people we know, or I know, right? Exactly. There's someone who's just been submitting pull requests, which is cool. Like, I've never spoken to him before in my life. One of them looks good. I still haven't gotten around to looking at the other one, but that's cool. **That's so funny. It just happened in that other code base that I was talking about that I'm taking now to the REG as well. Someone was like, 'Oh, I want to contribute. How can I resolve this issue for you?' And I was like, 'Hold on, we just opened the repository. We need to get our ducks in a row before we get to that point.' But that's really cool. And it really speaks volumes, I think, to both the look of the project itself and the quality of the code. But also, you know, the sense that there are people who are interested out there who definitely will connect. And you know, there's a potential of reach already. Like, having that tells me that there's an appetite for something like this. And we just need to figure out a good way to get to the users, I guess, and the contributors in some form or fashion. But you mentioned other people, people that we don't know. Are there any other people beyond the pull requester?** So, there is this PhD researcher from India who, it's called [Zeal Patel], gave that testimony. So I'm actually meeting him for the first time on Friday. And yeah, he's very keen to use DeepSensor, and you know, I've just launched it as well, pretty much. Exactly. So I guess we'll see if interested faces start trickling in. But of course, they'll see one of the first things on the code base like, 'This is undergoing active development. It's not at version 1.0 yet.' So, right, you know, it's liable to breaking changes, which might be slightly off-putting. So I think when we get to a point where it's like, 'Here's a stable release. We're not going to do any breaking changes anymore.' But to that end, I'm keen for the next few months to be a bit of a flurry of just trying to battle test it in different ways so that it will be as stable as possible when we reach version 1.0. **Yeah, do you have an idea of when that would happen? Or what would be required to get to 1.0? Or is that coming out of that process more dynamically?** It's quite dynamic, yeah. To be honest, I'm fairly new to versioning, but yeah, I'm using them as milestones in the issue. I'm just like, 'That feels like maybe a step after the next step. I'm gonna make that 0.3.' But yeah, I guess breaking changes is the key thing that distinguishes, in my mind, version zero from version one. **Yeah, sure. Like a major kind of change in the way that you would use it or whatever. I mean, version one is different in that sense, right? Like, you need to decide, I guess, when you're at that point that you're going to say, 'This is the point from which we then will break things in version two.' Or something like that. Something like that.** Yeah, it's like, 'We're not gonna change stuff that much anymore for a long period,' or something like that. **Yeah, yeah. But you've been... So, I mean, obviously, this isn't your full-time job just doing this, right? So, there you... I'm sure you're busy beyond compare. Like, how have you been doing this? Is this just a passion-forced/passion-powered project?** Yeah, it's a labor of love. I'm really enjoying moving more into engineering—research engineering—and I've kind of made it my job a bit. So, yeah, it's intense trying to do research and this at the same time, but they actually have a lot of overlap. Because like, I'm using this as the tool that I use to do my research. So when I improve the tool, then my research benefits. **Right. Because I'm curious if that would be something that you would be interested in. I think I heard you, but I might be wrong, hint about that in the talk as well, sort of submitting it to something like the Journal of Open Source Software or something similar to that. Because I would, obviously, then... you know, as... So I'm, I don't know, I didn't introduce myself when we... feels like we'd know each other already because I just looked at all the talks and everything, but I'm coming from a background of humanities research originally, but then for the past year...** And a half, and then like a year and a half as a software engineer, is that...? **Exactly, yeah, yeah, yeah, yeah. So that's the... you know, that's the... I'm coming from it at this from that sort of angle of having been a research software engineer and trying to figure out what that is and how, what the research component of it is, right? And I think that that's really this idea, right, that there's a... we're not just writing software for commercial purposes, but we're actually submitting it to peer review places. And it's almost like a research project in itself. Like, the code itself can actually be part of the research. It's not just the tool with which you do the research. So, in that sense, like, that's something that I've been thinking about as I've been looking at this code as well because I think you mentioned something about that in the talk, or in the Q&A, or something.** I don't know when I mentioned JOSS, maybe it might have been in a Slack channel, but I would like to do that. Because I think the review process would be really good. Because I don't have software engineering training. I'm learning as I go along, and you know, learning quite quickly because there's loads to learn and it's enjoyable, sure. But like, there will be legacy things just from like, I implemented this when I was a bit more naive. And therefore, you know, I didn't know about abstract base classes, which I just found out about today, and I'm like, 'Oh, I'm going to raise an issue that we should use abstract base classes.' **Yeah, yeah.** But yeah, just like through that review process of JOSS, I think would be cool. Yeah, I don't know how worth it it would be because it is also kind of like being reviewed by the world because it's open, and anyone can look at it. Um, yeah. **Yeah, that's what I was going to say. That, like, it's a difficult decision. If you're having trouble formulating version one, then it's definitely a big step to then also submit it to a journal and say, 'Hey, anyone who is reviewing for any of these softwares, have a go.'** Yeah, yeah. **Um, it's a different kind of commitment for sure. But it's good to know all of these things. And the reason why I'm asking about process is also partly because that's been a passion project of mine: project management. And I've become certified as a Scrum Master, and thinking about rapid prototyping, and implementing sort of sprints, and thinking about the best ways of structuring these kinds of things. So I'm happy to help you with that kind of stuff too, if you're interested in that.** That sounds perfect. **Yeah, and I think also, specifically, once these things are usually good—I've come to notice—when it is the case that you have been spending so much time yourself on doing a lot of the development and then all of a sudden there are three or four more people on board or you start having these pull requests from randos on GitHub.** Yeah, yeah. **It's like, 'Hold on, how do I fit all this in?' Yeah, exactly.** Yeah, so I have scheduled a fortnightly meeting. We've had like four meetings so far, four or five, and I kind of struggle with them because I don't know what... I haven't even defined my own thing that I'm trying, how I'm trying to run this. But the invitees are like people who have been using it already, people who have been contributing. I think Scott dropped into one. And it's more just being like a space for researchers interested in what this novel sort of ML paradigm could bring, or who are already trying to explore it. But I think it would be good to kind of focus... well, I don't even know. This is really something that needs to be discussed. Yeah, and just help with running this thing would also be really good, and I can invite you to the calendar invite, so it's in your inbox now. I was waiting for this chat to tell you about it. **Absolutely. I mean, that would be a really cool way of engaging with, I guess, the people who would be using it in the end and all that as well. How did you... So you just started that, you just set up a meeting, and then let people come essentially?** Well, yeah, it's a calendar invite. It's a Zoom calendar invite at the moment, so I was just like, 'We need to start actually speaking together.' Yeah, yeah. But it needs to be defined more, and I'm keen to chat that through. It's too shambolic at the moment. It's just, 'How do we spend the minutes? Who's it for? Is it for the users, for the contributors?' So far there haven't really been that many contributors, so it's more just been like me being like, 'Hey, here are the updates to DeepSensor. Here are the open issues. Should we chat about your research? Let's see how we can improve Deep.' So, quite informal like that. **Right. So did you post that on different Slacks, or what was the...** Just put like a personal invite sort of thing. **Cool, um...** Do you think we should have a Slack channel within a dedicated workspace? Not dedicated, but like hijack some workspace and make a Slack channel for DeepSensor? **Probably. I'm, um, all for doing that. I think it's better to do it in existing, like I said, in existing spaces than creating its own. I would imagine that would be a good place for it. Because we, yeah, we did that with MapReader. We went a little too fast, I think, to our own organization, and then it's like, 'Hello? Hello? Hello?' [echo-sounding effect]** That was a really good one. **Yeah, you just feel like it becomes a little sad. And it's better to have, you know, a little group within a larger organization and then be able to kind of connect out. And you can always move to a separate organization once things happen, you know, on a different scale. But I think AI4Environment would be the place. I don't really know how many people are on there; I haven't checked that. But 382 people, so like, that's obviously a fair amount of folks. Probably a lot of the people that we know, but it's, yeah, I mean you're sort of riding on the coattails of other people being invited and all of that. And so you could pick up random people who would be interested in something like DeepSensor through permeable walls or something.** Word of mouth? **Yeah, yes, exactly. That's cool. That sounds great. I just realized that we, I did set this meeting to 40 minutes, and so we have reached the end of the meeting.** I will have to drop off actually shortly, yes, just to wrap something up. But this has kind of assuaged— is that a word?— a lot of my fears that I had, which is that, like, yeah, I'm just gonna struggle to maintain everything. I've kind of released this beast and then I have to tame it and it'll be too hard. But I think, you know, by working together and coming up with smart ways of having public-facing information, it'll be fine and get more people involved in contributing. **Yeah, that's all that's needed, for sure. I think that sounds like a good idea. Obviously, this is a starting conversation, not a final conversation, so...** Yeah, exactly. **Exactly. And, like you saw, I have a lot of questions that I'm gonna think more about and probably will reach out to you about. And maybe we can get another chat on the calendar at some point. But it sounds also like that chat that you have fortnightly might be a good place to start having a bit of a bigger conversation as well. And once we find out about the Force Multiplier, then that's definitely also something that will be good to know more about.** Definitely, definitely. **Um, yeah, great. And if you wanted to loop me in on that, also, by the way, feel free to do that. I don't know if you have something written about the scoping for the Force Multiplier, but feel free to forward whatever you have about that as well.** Yeah, I'll send you the one-pager that I put together. **Perfect. All right, okay, I'm gonna let you go to finish up what you need to finish up. I probably should wrap up my day as well. Great to see you, good to talk, looking forward to things—spectacular things.** Yeah, it's amazing to have you on board, and yeah, thanks for your time. I'm looking forward to where we can go with this. **Cool.** Enjoy the sun. **Yeah, um, too late.** All right—bye. --- ## DeepSensor conversation preps ## Relevant links - [DeepSensor on GitHub](https://github.com/tom-andersson/deepsensor) - [DeepSensor gallery](https://github.com/tom-andersson/deepsensor_gallery) = demonstrators, use cases, and videos --- ## Tom & the team #### Could you share a bit about your story and what led you to your current research? How long have you been working on it? What has the journey been like? - (Talk available [here](https://www.youtube.com/watch?v=v0pmqh09u1Y) and [here](https://www.youtube.com/watch?v=MIHNyKjw204) -- the latter is more interesting) #### Are there other members on the team that you're working with? Who are they? - Looks like it's only Tom. - Some other "collaborators" mentioned in recent talk: Wessel Brinsma, Paolo Pelucci, Rich Turner, Jonas Scholz #### How did you start working with your current partners? - ... --- ## DeepSensor #### What problem does DeepSensor solve? (or what applications is DeepSensor used for) and currently, how does DeepSensor function? - DeepSensor is a Python package designed for modeling environmental data with neural processors. - optimise the placement of environmental sensors - has active learning functionality, which could help in iteratively improving the model based on new data - scalable and extensible: design allows for scalability to high dimensions (e.g., adding depth and altitude as dimensions or handling hundreds or thousands of observational variables). It is also designed to be extensible, meaning that new models can be integrated into the package. - lowers the barrier to entry for both environmental scientists and machine learning researchers, thereby encouraging more collaborative and cross-disciplinary work. #### What metrics are used to determine when the problem has been solved? - ... #### What are the next steps for future DeepSensor development and what is the proposed timeline? How much has been scoped? - Recent talk mentioned a challenge in scaling the model to high dimensions, such as adding depth and altitude as a dimension or handling hundreds or thousands of observational variables. - Future work might involve applying transfer learning techniques, as mentioned in the context of transferring from densely to sparsely monitored regions. - Potential for integrating with other machine learning architectures, such as graph neural networks or transformers, indicating an openness to expanding the types of models that DeepSensor can work with. - Recent talk notes that these models start from “square one” and may require large amounts of data to train effectively. Future work may focus on making the models more data-efficient. - Recent talk describes galvanising research progress and building a community around this open source software, leading to a positive feedback loop where software improves research, and research, in turn, improves the software. <-- This seems to be a place where RAM can come in! #### What resources are required to complete development? - ... ## Stakeholders ### Partners #### Who are stakeholders (people and teams) that you can identify *within the Turing Institute* that you are currently working with? - ... #### Who are stakeholders (people and teams) that you can identify *within the Turing Institute* that you'd like to work with? - ... #### Who are the partner organisations beyond Turing you are working with? - ... #### What are their expectations/motivations for the project? - ... #### How do you feel about the current state of the partnership? - ... #### Are there other partners you'd like to explore working with? - Other similar research institutes focused on data science, artificial intelligence, or environmental science? - Environmental Agencies or Organisations? - Meteorological Organisations? - Universities/academic institutions? - Open Source Software Communities? ### Contributors #### What kind of expertise would you say is missing from the project team, if any? - ... #### Have you identified any outside contributors you'd like to work with? - ... #### Have you reached out to specific contributors (for instance, asked for volunteers, or similar)? - ... ### Users #### Who would you say are the current users of DeepSensor? (be as specific as you can in terms of roles or company function) - ... ## Impact opportunities #### Who would you identify as being most affected by the problems that DeepSensor solves? How? - ... #### Who would you guess are in a position to start using DeepSensor now to solve their problems? - ... #### Have you (or anyone in the team) reached out to those people/organisations? - ... #### What message do we want to convey to those people? - ... #### Have you (or anyone in the team) identified ways of conveying this message to the potential audience for DeepSensor? - ... #### Where do you hope your research will end up? #### What does impact and/or success mean to you for DeepSensor? #### What are you doing now to try to achieve this impact/success? #### What is preventing you from achieving this impact? --- ## Ideas - Add a Sphinx docs for DeepSensor, with auto-generated pages from docstrings, that can be presented in a web page through GitHub Pages (easy) - Add a website presence for the DeepSensor Gallery, for SEO and niceness (easy)