I love these guys! I helped advise them on their crowdfunding campaign a few years ago. One of my favorite projects.
Hey Topher! Are you guys cross checking this with daily satellite imagery? I met with a guy who's company gets a full HD shot of the earth every 24 hours and they'd be happy to give you guys access to the visual data (they also train algorithms on it to detect all sorts of activity).
Planet does not image the ocean in its entirety due to the difficulty in geo-referencing (pre-user imagery processing pipelines use ground control points, usually just shiny objects with a known location, to locate the image on the earth, and this is hard in the ocean), but they do image focal regions for purposes similar to those you mention (vessel identification + tracking). I'd also imagine this choice has something to do with preserving satellite operational lifespan (can turn cameras off when not being used).
I can ask, but I assume so. The guy I spoke with was doing things like algorthmically measuring shadow angles on crude oil storage tank covers to determine supply stocks and then selling that data to commodities traders. They were able to get incredible detail along with really robust processing of huge datasets.
Doesn't NASA have a dedicated satellite with API access for this type of data? (Monitoring weather, tracking deforestation, crop changes, etc). I believe its called VIIRS
Purchasing a before and after image of a specific logging site isn't cheap once you've factored in minimum order sizes, but it isn't unaffordable. Purchasing images to look at deforestation on a national scale (if you're looking at individual trees, not woods; open data will give you the latter) is millions of dollars per annum, but worked very well where tried to prevent illegal vegetation clearing in Australia.
Thanks for checking out Rainforest Connection! Seems that there are lots of [appropriate] questions about how well this works in the forest, and how our partners on the ground follow up on alerts. Here’s a recent intervention in Peru that sums up the process:
https://twitter.com/lilitjejauregui/status/97585519352188518...
This is a really cool application of this technology, great work! Do you have any blog posts explaining the types of models you're using or open source repos that one could contribute to?
(Asking as a ML/Software Engineer who's pleasantly surprised and yearning to contribute to something like this!)
Thanks! We are a relatively small engineering team and we were mainly focused on improving our system. This took a lot of effort and time, we just haven't had a chance yet to think about open sourcing or blogging. Our days were focused on coding. If there's interest in blog posts, I could write up some of our technology. Just let me know whether there's an interest and what you'd like to know.
I know that the US National Park Service Natural Sounds/Night Skies division has an incredible about of environmental audio data that's begging for automated analysis. This could be of much use to them. They do analyse their data, but it's largely been manual [1].
That's very interesting. The Rainforest Connection system has the ability to detect more than only chainsaws, so it could be really interesting to learn more from the datasets; we'll look into that. Thanks so much for sharing!
Just signed up for your newsletter! I understand you don't want to open up the audio fingerprinting details, but everything else would be very interesting. I am interested in how do you get an old Android phone to run some your APKs headless, how do you update them OTA, how do you power them with solar (I guess you have an external power bank) — some basic questions if I want to put an old Android phone to my DIY project use.
I don't use Android but just did some googling and only have these ideas: roll your APK updates over a custom F-Droid repo, use SSHDroid to log into the headless machine, and/or use one of the DEX patching libraries. How far off are my thoughts?
Do you setup wifi hotspots in the forest for this devices to send data back? Solar chargers is it universal for any android phone? what happens if phone is unable charge and shuts down?
are those solar chargers available for sale to use in other forest projects?
Also, I recall hearing about how some people or villages opt in to deforestation because they are offered a significant amount of money which can be used to feed their kids and provide an education which may help their communities eventually escape the poverty trap. Do you know anything about this, and if true, do you know of any groups trying to help?
Where do I begin if I want to set something like this up?
As an experiment - my neighborhood in India has a problem with loud motorbikes that are illegally modified. Hypothetically if I wanted to use this approach (there are probably simpler ones in this specific case), where do I start?
Do I just start recording gigabytes of audio data every day and start doing some kind of FFT on the data and manually tagging cases of loud bikes?
After that I suspect I can use one of those ML cheat sheets to figure out what is the best model for this kind of thing and then I just send it data via the TensorFlow library?
On the surface of it it feels like I'd be spending hours of processing time converting the audio data and hours of manual tagging before I get anywhere close to a solution. Is there a quicker or smarter way to go about this?
I'd start with the end goal. Is the acoustic signal from the motorbikes likely to be traceable to particular bikes? If so, would the local police actually be interested in following up?
There is value in quantifying just how noisy the bikers are and for just how long they disturb the neighbourhood, but you'll probably need the perpetrators to be people the police are able and willing to stop to get anywhere.
Yeah I've already spoken with the police they are down with nabbing these folks if there's video evidence.
I could theoretically leave a camera running or simply trigger on high enough volume but for assuming I wanted to do it this way I'm curious about the best approach.
I’ve been thinking a bit about wearable computing and pervasive encrypted communication. I think personal encryption could become more necessary in an age of deep learning enabled fake video and audio. Wearable computing could take readings from your body and environment as a sort of timestamp and fingerprint. It could also enable an easy way to type messages to others privately.
I'm not too familiar with ML, do you have to "train" tensorflow as to what audio in the data is "definitely a chainsaw" and which is "definitely a logging truck", etc etc? As in, you should keep doing this every now and then as you get more and more data, at least in the sense of flagging false positives (so as to unlearn that particular sound).
Yes. In the video there is a short snippet where the audio is shown as a Fourier transformed image on the screen and a user is annotating the image of the sound using red boxes. This is a part of the process to train the ML model to recognize chainsaw sounds vs. other sounds.
Thanks for noticing the spectral analysis. We put quite a bit of work into the training system. Besides the base-level Fourier transformed images, we also have a UI for partners who can easily report if an alert was correct or not which also feeds back into the system.
I also work with non-speech audio and I'm curious: Do you use pure DFT:s as inputs to your models or do you use mel-energies or MFCC:s? What kind of models do you use? Since there is not that much variation in the sound of a chainsaw I suppose either a regular fully connected or convolutional neural network?
Love what you are doing and I would love to see a technical blog post about how you work with audio!
That's a great question. Actually, one of the sounds that are pretty close to a chainsaw are mosquitos that are circling around our microphones due to the Doppler effect. We found ways of dealing with signals that are close to chainsaws by aggregating multiple models and also a time-based analysis. The system can draw causal/correlative conclusions such as a vehicle is usually present before a chainsaw. If there's no vehicle, the likelihood of a chainsaw goes down and the chainsaw model must be highly confident before we sound an alert.
How do you quantify the confidence of your model? Do you use a Bayesian model or just the log-likelihood? Because the latter can act strangely in some cases.
I know this is a digression from the current discussion on how well the devices work, but as a stats student who just learned about estimating using log-likelihoods, could you give some more info on how that is inferior to the Bayesian model (since I've heard the exact opposite is true)?
The problem is that neural networks trained using maximum LL do not return calibrated probabilities, using e.g. the softmax output as 'confidence' of a model tends to result in overconfident predictions, take a look at adversarial attacks on neural networks for an extreme example: https://blog.openai.com/adversarial-example-research/
I imagine it would depend on the lifespan, and travel patterns of a lyrebird, no? Eg, bird is making the chainsaw noise weeks or years after the loggers have gone, possibly in an entirely different area.
This technology is really cool. My question would be, "What happens when your AI detects chainsaws or logging trucks?". Do you contact some policy authority, some international authority? Also, why are their cell phone towers in the middle of the jungle? What about areas deep in the wilderness where no cell towers exist?
That's for small time abuses. What happens in the case of a full blown illegal logging operation involving a entire company? What about cases where things get violent? It's not uncommon for mining operations in the Amazon to end with natives getting massacred, the loggers probably not unwillingly to do that to anyone who gets in there way. When you come across a full scale logging op, do things get complicated?
Exactly. In that case they arrested a couple of poor peasants, but the people/organizations that both financed and enabled it walk free.
Like with drug labs in the jungle, all this cool technology is great for detection but ends up hitting a brick wall of corruption when it comes to actually solving the problem.
Great points you bring up and indeed things do get complicated as you face a bigger and more organized operation. This is the case in parts of the Tembe territory in Brazil and it is the reason why we work with many partners and agencies on the ground to ensure that the data we capture is used wisely.
Had the pleasure of working with their engineering team last year as part of a pro-bono data consulting grant that a nonprofit I work on (http://www.deltanalytics.org/) provides.
Excited to see how these systems are integrated into prevention / intervention and other applications for these systems beyond detecting illegal logging.
How about ultra-low power microprocessors coupled with capacitors, solar cells, and MEMS microphones, that can use a continual Fourier analysis to identify gunshots? The internal memory could be kept so small, as to preclude the possibility of recording someone's voice for longer than one phoneme. These could then be sprinkled around someplace like the gang territory parts of Oakland.
Being a non-native English speaker, I had to read the article to finally understand this isn't about chainsaws with embedded logging devicel, like 'black boxes' in cars.
> We’ve hidden modified smartphones powered with solar panels—called “Guardian” devices
Question, how is the electrical PV power budget vs load budget calculated for these? Having implemented PV projects ranging in size from 300W to 30kW, I'm very interested to know what their predicted Wh per day generated from the solar cell is vs. what the load will consume (what's the actual load of the cellphone, how often does it wake up every day, etc).
Great question, and one that’s of particular importance in our design. Many IoT-type systems make use of on-device data analysis in order to minimize the most energy intensive process: data transfer. Our hardware focuses instead on getting full audio streams up into the cloud in near-real time, continuously, 24 hrs a day. This requires an exceptional amount of energy, as well as the ability to pool energy to continue operating at night and during poor weather. That said, we’ve really overcompensated on energy capture and storage—we operate under the assumption that we should be able to support 1.5 watts of continuous power draw 24hrs a day (normally it’s closer to 50% of this). But that means we try to be able to generate 35-40 watt-hours of solar energy for each day (on average).
This is further complicated by the shady/infrequently lit conditions of the tree canopy. So, when direct light shows up (sometimes for moments or minutes only) we want to be able today capture as much as possible (high charging amperage and ample solar panels).
Our batteries (LiFePO4) must be able to support this, and not decline over 1-2 years of use being fully discharged and recharged each day).
We also designed a type of solar array that is specifically intended for the tree canopies. If this is interesting I’d be more than happy to elaborate more!
That is a really good answer, and thanks for taking the time to write it up. The difference in instantaneous wattage produced by good quality 156mm monocrystalline Si cells is huge in direct sunlight vs shade. You can take a modern 60-cell, 300W rated panel (STC rating) and hold a piece of cardboard a meter away from it, to temporarily shade it, and see a huge difference. What size of PV panel are you using for each node, and how many/what type of cells?
I'm assuming that for cost reasons and economies of scale that you're using some sort of standard off the shelf PV module that can be acquired in bulk, at low cost, such as a 156mm poly x 54-cell module. Or maybe something a bit smaller than that.
1.5W load x 24 hours x 31 days = 1116 Wh per month
For a typical node what is your calculated PV production in Wh per month? Taking the 40Wh per day figure mentioned above, that's 40 * 31 = 1240Wh per month production goal from the PV. Will of course vary with mounting location and shading, but I'm sure you have a floor figure that you don't want to go below, or batteries will reach a LVD threshold overnight and everything will shut off.
I'm also making an assumption that per-node cost is somewhat of a concern in general. For very small PV setups a decent charge controller (that can handle up to the max of one medium sized panel) is not very expensive. Are you using an off-the-shelf $65-75 chinese small MPPT PV charge controller with voltage levels set up to match your LiFePo4 battery setup, or something you've designed yourselves? something like this? https://www.amazon.com/DROK-Controller-Converter-Regulator-C...
Are there methods to change the noise of a chainsaw, as there's money to be made in illegal logging - they'll have the incentive if they're organized, so it would be quiet enough or different enough to potentially sound like it's "outside" of a protected area or not a chainsaw at all?
If and when they do that we will find a way to adapt and detect. The beauty of a real-time system is that you can make changes/improvements remotely without having to go out in the forest again and swap devices out.
In fact, the forest is a very loud place and yet we are still able to detect the sound signature of a chainsaw.
I was thinking and wondering how reasonable as a "next step" in protecting forests to potentially require GPS tracking on all or certain types of chainsaws - it feels like an absurd thought on its own, though it potentially helps solve the problem.
This could then extend tracking to imagery via satellite - via geographic and density changes being detected, knowing where the GPS-tracked and legal work was being done, being able to then eliminate those areas from as high of scrutiny. Perhaps the GPS tracking is then not necessary if you can accurately compare before/after imagery, though could be a useful tool anyway for management to know how efficiently equipment is being used, if it's at where it should be and such.
This reminded me of a story my father once told me. I assume he heard it because his father was part of the Ministry of Forest in Canada a long time ago: I believe it was during a world war (not sure which one) when I could imagine lumber would have been in constant demand. There was a bit of a trick and literal loop-hole that truckers would use. Before being discovered there apparently was a long route where truckers would go through without actually dropping off their lumber - after having gone to the weigh-in point to get paid. They'd continue to drive through the back to return another day with the same load to get paid again; a long drive apparently but still was a money maker..
Obfuscating it with continuous loud noises would probably be easier than trying to change the sound of a the machine itself. A lot of the noise comes from the wood being cut, so even if you could totally silence the motor there is still that.
I’d just show up with a large cell jammer on the roof of my logging truck. Probably cheaper than continuously modifying the sound of trucks and chainsaws?
Until of course they’re using decent newer phones that can do edge inference
Yeah it's pretty naive. Google trying to make the world a better place would actually be illegal and would place the managers in jail for defrauding investors. Google's mission is to make as much money as possible, typical tactics include creating a good corporate image.
That a company's mission is to make money for their shareholders does not mean they pursue that mission to the exclusion of all else. They can also pursue side goals, from making the company a nice place to work at, to giving back to the society which allowed them to exist and grow in the first place. Some of these goals may have little to no tangible results on the bottomline, which is ok as long as they don't compromise their main objective. You seem to like to view the world through a cynical lens which is no more true to reality than a naive lens.
First of all: no jail. This is civil law, not criminal.
Second: it's pretty hard not to have an excuse to do anything you want if you're the board + management. Google, and most other companies, give to charitable organisations, for example. "Now", you'll say, "they're just doing this for PR".
Maybe... But using the PR argument actually works perfectly fine for anything altruistic a company may do.
Unless management explicitly states that they're harming investors for the public good, it's almost impossible to run afoul of the law. Sick pay that exceeds the law? Important for employee retention (and PR). Match your employees' donation to any certified charity? Important to create a community (and PR).
If a publicly owned company started to contribute too much money to charity, they would certainly be subject to shareholder lawsuits and a proxy war over management.
Sick pay is easier to do because it's related to retaining good employees.
But generally, as Milton Friedman famously wrote, "the social responsibility of corporations is to increase their profits".
Defrauding investors yields jailtime more often than not. Depending on the particular case is might be a civil case, but even civil cases often result in incarceration.
I'm not going to argue with the rest of your post because your belief that a company might be doing "anything altruistic" is just sad for me, perhaps laughable for others.
The mission is to organize & make available the world's information... which coincidentally is equally compatible with increasing shareholder value.
I think most people dramatically underestimate the amount of resources Google invests in all sorts of programs (including R&D, charitable donations, education, partnerships and so much more) because 1) they don't use it as a negotiating tactic, and 2) the cash cow of the ads business provides an enormously long leash for experimentation and.
Even with Apple, which has more than double the cash (and cash flow), doesn't have this flexibility because of its business model. As a hardware company, volumes, revenue & margin are predictable and anticipated, baked into the stock price with an assumption of -- for the last few years -- dividends. Amazon is at the other extreme. Until very recently, as a result of AWS' runaway success, Amazon had posted negative-to-zero profits for its entire history, reinvesting nearly 100% of free cash flow into business expansion.
No matter what you might think of Google, among the huge tech companies they're unique in their ability to spend enormous sums on things that absolutely do not directly increase shareholder value. The belief within the company, and within the investment community, that some of these moonshots & experiments will eventually stimulate stepwise returns are why they continue to run this way. It's cultural, and ultimately, one of the foundations of "googleyness".
Have the creators of this project considered the privacy implications of capturing, recording, or analyzing audio from the rainforest? What happens if people are walking near a microphone and a conversation is picked up by the microphones?
I realize that the audio is probably analyzed for the sound of chainsaws, and then thrown away, but there's still the potential that such a system could be misused. So just wondering what people think about the ethical issues related to this technology.
Not really answering your question, but in the interest of protecting a critical ecosystem I think some theoretical privacy loss is an acceptable trade off. Privacy is not the be-all-end-all.
I think you've a bit over-analysing it. I doubt the conversations are even audible at the places where these phones are attached. Then there also is the fact that no one will probably even get close to these for most of the time.
While reflecting on this--and ethical issues aside--it did occur to me that there's probably a lot more data you could collect from the audio gathered in the rainforest, i.e. what animals are active in what areas of the rainforest and what times, weather data such as intensity of rain or wind, volume of animal activity in a particular area of the rainforest, etc.
Someone more familiar with satellites can correct me, but from my understanding you should be able to detect deforestation from real-time satellite data. You alert on changes in the level of greenery in the images. The indicator might trail behind detecting chainsaw noises, but it wouldn't require the mass distribution of detector devices.
one of the problems is that true near-realtime satellite imagery (90 degree angle direct overhead, visual spectrum or IR spectrum to detect tree cover) is quite costly, in terms of $$$ per square km, if you want images that were acquired recently. The older the data is, the cheaper it gets. This could be a huge recurring cost to buy new satellite imagery.
If you want almost-true-realtime satellite imagery for land-use law enforcement purposes, you're looking at a project that required the financial resources of a nation-stage IMINT apparatus (US NRO) for example, with dozens of polar-orbit, LEO satellites. The best you're going to get from commercial sources is a couple of days old, up to a few weeks old.
This is being done by many [ex. 1], but you've identified the challenge: often it's too late to catch offenders once the satellites have seen the damage.
The title is asking for attention unnecessarily since the neural network will not be able to distinguish between "legal" and "illegal" chainsaws. It should just say: Listening for logging chainsaws [...]
This is a super interesting tip — thank you, we were not aware of this (it is reminiscent of tracking of shipping vessels). Do you know of where and how we might be able to make these data available to local partners, like the Tembé tribe, and others? Do you have any contacts at Deere Forestry? Thanks in advance.
What would constitute a non-fraudulent blockchain ad?
All ICOs are all designed to be the same Ponzi-Pyramid scheme, as how they're incentivized - which is what makes Bitcoin, Ethereum's Ether et al in the same boat.
Most ICOs are speculative but not Ponzi/pyramid schemes. They do not borrow against previous investors' funds and they do not have referral DAGs. Do you consider Facebook's IPO a Ponzi/pyramid scheme?
I don’t think so. Google is massive and people that work there have been doing lots of different research for a very long time. OP has nothing to do with the past days of Facebook stuff IMO.
If they're trying to get away from privacy-related bad PR, maybe "oh, by the way, we're capable of bugging the entire rainforest and analyzing the take in real-time" isn't the best approach.
The response to counter paranoia is to have trusted organization - that you can learn to trust, needing to learn how to trust as a pre-requisite - and regulation to counter bad behaviour and be proactive where reasonable.
Any violent act, no matter the size or whether it looks like or is a so-called legitimate reaction/response should be a signal/flag for response. All of these systems can be used for good or bad, they can excel at either - the broken piece will always be a lack of trust and relationships with community. This is the fight between good and bad/evil, between light and dark. The nervousness caused by bad/evil should be used and funnelled into strengthening oneself, one's resolve.
There may always be people trying to take advantage of these systems throughout history (perhaps at no fault of their own, we're born into the environment we're born into), however we can design systems for the environment and people to be healthy - health including being attached to empathy properly and to develop a practice of non-violence within society, and while educating to help people understand the value of these surveillance systems like tracking illegal activity in the management of Earth's resources - whether forestry, fishing, pollution; we're all fucked if we don't manage these. Not being a responsible being wanting to work together to manage these resources is very short-sighted and shallow thinking (primarily because of people stuck on scarcity/survival thinking), because we literally could all die if we don't manage these system. With economies of scale and automation we can do this - we can succeed. We can give an increasing higher quality of life to everyone than anyone on Earth has likely lived thus far. This isn't fiction, this is truth.
Stephen Hawking before passing was worried capitalism will get the best of us though -- that is his rally call for us to stand together, to align, and manage the negative impacts that a capitalistic-transactional system requires; http://www.businessinsider.com/stephen-hawking-final-reddit-... - the way we can honour this contributions to society is to, for those of us who are able, to stand and fight together. His passing on Pi day, the day Einstein's birthday, was no mistake - it's a bright beacon of light, the rallying call. Pay attention, stand guard.
They're certainly in a prime position to try to say that they're not eeeevil like Facebook.
They can sort of imply: hey, what did Google search, Gmail, maps, Android ever do to you? Microsoft and Apple are in a similar boat. It'd be sort of comical, if Google ends up benefiting from failing so spectacularly at social media.
Amazon has a problem of terrorizing other competitors at the moment, that's primarily who is upset with them (spurring endless media stories about Amazon's growing power). Consumers mostly love them.
Exactly. Not to mention that some other guy in this thread said that in Brazil now all forestry equipment is sold with GPS tracking, so there's literally zero need for such a device regardless.
If they want people to stop chopping down forests they need to provide them with a differnt, viable source of income. If I was living in deep poverty and there was a forest near me I'd chop it down, no question about it.
This money should be invested into fusion or solar research - if energy is cheap- somebody will find a vay to convert it into fast vat grown rainforrest- and voila->
Unemployed drunks protesting before the automatic growthlab and sawmill.
Its very hard to prevent behaviour thats economical. Its way easier to subvert the economics behind that.
If you want to save nature- compress agriculture into vats. If you want to compress agriculture into vats and buildings, you need cheap energy.
Great, now you can't escape Google's tracking even in the middle of the fucking jungle. Those bastards will triangulate your position from the sound of your footsteps and send BigDog to slice you up.
Hey Topher! Are you guys cross checking this with daily satellite imagery? I met with a guy who's company gets a full HD shot of the earth every 24 hours and they'd be happy to give you guys access to the visual data (they also train algorithms on it to detect all sorts of activity).
Anyhow, I'll shoot you an email.