I'm sad that this is happening to an app that's useful to its users, but the reality is that scraping is legal, always possible, but difficult.
This particular case is a bit harder since it's not purely using public data, but may still qualify since it's likely scraping with legally-obtained credentials.
I know of businesses (scraping for ride-sharing, scraping for business intelligence for retailers, scraping from LinkedIn - see HiQ Labs v. LinkedIn) that have continuously succeeded via scraping in ways that large businesses oppose.
The key is: you must make enough profit to justify dedicating engineering and legal techniques to defend your scraping.
- Scraping public data is legal, as affirmed by the Supreme Court in Van Buren v. United States [1] and HiQ Labs v. LinkedIn [2]. Defending yourself or suing the data owner in court are both expensive though
- Defeating anti-scraping via technical means is pretty much always possible, but can be costly depending on the scraped site's technical expertise and value in keeping their data private. The benefit to you must exceed the cost to you, and ideally should also exceed the cost to the data owner
- Mobilizing PR and internal resistance may also be effective, but it's usually hard to have outcry from a large enough group to change an organization's policies. In this case, the union can push for it, but AA may try to withhold improvements until the next set of union negotiations
> the reality is that scraping is legal, always possible, but difficult.
> This particular case is a bit harder since it's not purely using public data, but may still qualify since it's likely scraping with legally-obtained credentials.
No, it's easy: they're employees, they can be told they're not allowed to do that. Doesn't matter if the app's legally allowed to exist or not.
They can always tell their employees they're not allowed to do something, and punish the ones that do. I think it's an important distinction though that whatever they tell their employees, the app isn't doing anything wrong in a legal sense. So still legal, possible, and difficult.
And I don't think there's anything illegal about the employer making it more difficult; that may well be the cheapest/easiest way of stopping employees using it.
There's nothing illegal about it, but that doesn't mean that making life harder for your employees trying to make sure the hours they're working are legal is moral.
EDIT: I welcome anyone who wants to justify this ethically.
I'm not sure I interpret the article or the parent comment as saying American Airlines isn't allowed to do this. They're just making life harder for their employees and dont seem to be addressing the problem they're working around. It's just a little whistleblowing that they're a shitty employer.
I think what the parent comment is trying to say is that their description of their approach here as "sophisticated bot detection" is a little bit like someone calling me a hacker because I have my terminal open during the flight. There is an intentional use of words here trying to make the app developer sound like the bad guy.
You know, employees are not actually property, and there are actually limits on what an employer can tell an employee.
Everything an employer might possibly try to say about using any other software or tools to collect, handle, and redisplay "their" data, applies exactly the same to a blind employees screen reader.
Hell it applies to glasses.
Thank deity for blind people and other disabilities making it actually illegal to be as huge dicks as some companies would be if they could be.
I do not understand the the desire to even try to defend AA's position here, but am glad it's a failed attempt at least.
I'm not sure why it reads as 'defending [the employer]'s position', but that's not my position, I don't care at all. (I'm not American, I may very well never have anything to do with the airline even as a customer.)
If you are its employee, jolly good luck to you with your 'well if I were blind what I've been provided with while not blind would not be adequate and I might need to use a different tool to this which works similarly' argument.
"I'm not sure why it reads as 'defending [the employer]'s position'"
Saying that the employer has the right to dictate those terms is literally and explicitly doing nothing else but defending their position that they have the right to dictate those terms.
Presumably the non-public data is being scraped using the employees' credentials (i.e. username and password).
It is perfectly reasonable for an employer to have a policy which states, "do not give your work username and password to a third party." I can't imagine a court ordering otherwise.
Providing an API for this data is a non-trivial amount of work, involving significant technical and compliance challenges. Employee schedules would be useful as a signal for trading in AA stock. How do you enforce that the third party is properly protecting that information, e.g. during SEC-mandated blackout periods around earnings?
The union might be able to negotiate for AA to hire lawyers and IT staff to work on such an API, but I really can't see the employees being automatically entitled to it.
If the scraping is happening "on-device", though, then they're not providing their details to a third party. They're simply accessing their schedules. Otherwise, pulling up their schedules in any web browser would be considered giving their credentials to a third party since that's basically what's happening here. It would be like logging in to the aa.com employee site and then installing a Chrome extension that reads the page that was downloaded. Nothing is given to the Chrome extension in terms of credentials, only page content.
It might be possible to build this app in a way that none of the information ever leaves the device. I would be very surprised if that was the case here.
Most large IT departments have a list of approved browsers and browser extensions. The scenario you described would fall under the same policy. If Chrome uploaded the content of intranet web pages to Google, I expect it would be banned as well.
Not only is it possible to build it this way but I think it's far more likely that it already is built this way. Since the app is pulling up schedules for individual users, there's no benefit to scraping the info on a server or caching any of it as it would be unique for each user. There's no reason for that info to leave the device. The content is pulled, formatted, and then displayed in a style that matches the rest of the app. This can easily be done on-device and would be less efficient to do off-device.
>Most large IT departments have a list of approved browsers and browser extensions.
This is completely irrelevant considering this is being done on mobile devices. On iOS, at least, it's all webkit and done within the app itself. I was just using Chrome as an example for how this process is done without sending the credentials to a third party. Unless the company wants to ban people checking their own schedules, there's no way they can stop someone from logging in to a web browser and having the content scraped. As an example, let's say they only allowed Microsoft Edge as the "approved" browser and they didn't allow any Edge extensions to be installed. The user can still pull up the page in Edge, save the content once it's loaded, and feed the folder/HTML file to the app to scrape the content. There's literally no way for them to prevent this other than by severely obfuscating the content (e.g., randomly adding invisible characters into strings to prevent string searches or adding bogus HTML elements to prevent searches for element patterns) or ceasing access to it completely.
Looking at the privacy policy[0], the app does send some information to backend servers, specifically "flight related information." If the employees' "flight related information" is not publicly available, I can see AA having a legitimate issue with it.
But the scraping does appear to happen on-device, and it claims the password is not transmitted, so that's better than I initially thought.
That would make sense for that context, though, since they're likely providing additional info regarding flight information and, according to the policy, only for flights that are saved to the user's account. That would imply that things like flight numbers are stored in the user's account which makes complete sense and wouldn't necessarily be anything other than public information.
>It might be possible to build this app in a way that none of the information ever leaves the device. I would be very surprised if that was the case here.
I'm doing some similar stuff to automate an app for personal use, I might at some point turn it into a paid for app, when I do so I would actually have to redesign the application to send personal information off the device. Which I suppose I would not do.
Airline workers are covered under the Railway Labor Act in the US. They can’t just strike because they’re unhappy. There’s a long drawn out process before a strike can happen. See recent threats of railway workers striking in the US.
>No, it's easy: they're employees, they can be told they're not allowed to do that. Doesn't matter if the app's legally allowed to exist or not.
They're unionized employees. Someone running a company looking to make their life harder for no reason needs to think five times before they start making arbitrary and baseless demands for changes in policy. It could end up costing you tens of millions of dollars because you forgot that employees are still people and your demands will be met with demands in return.
> The key is: you must make enough profit to justify dedicating engineering and legal techniques to defend your scraping.
It also works if you have philanthropic, non profit, or unconventional backing to pay for these defensive resources. If this app is providing substantial benefits to the AA crew around scheduling and QoL, their union might consider providing some backstop/support.
If memory serves, FA unions often use seniority-oriented contracts. The more senior members will tend to be more active and better-represented among union leadership. Reserve members are often more junior.
Putting on my cynical prick hat for a moment, I would guess the union as an institution is far more willing to throw the app-oriented concerns of the junior members under the bus than the health care and pension concerns of the senior ones.
> The key is: you must make enough profit to justify dedicating engineering and legal techniques to defend your scraping.
That's why web scraping is a huge SaaS market these days (I'm part of one too @ scrapfly.io).
Loads of our customers are tiny businesses and entrepreneurs that could no way afford the engineering effort required to scrape any of these websites and honestly empowering small folk against these giant, untouchable corporations is the best part about my job :)
Playing devil’s advocate here: this is not the same as scraping LinkedIn data. Linkedin data is public. This app requires a login info from a flight attendants to scrape their schedules. When you try to log in, you can choose to login as public or as a AA flight attendant. It sucks, but I also understand why a company may be unhappy, that a third party handles credentials and accesses internal data. What they can:
- build a 3rd part integration API, which opens up a whole can of worms. Not many tech-first companies can do it right, for an airline it’s a very challenging steps.
- build their own, but they already failed there if their employees turn to 3rd party
- ignore and let it run. This is basically unauthorized access to go and hope that the guy names Jeff won’t screw up.
- deny and prevent access. This is probably technically the easiest and safest from legal standpoint.
> This app requires a login info from a flight attendants to scrape their schedules.
So? If the flight attendants have provided their credentials to the scraping software, they have essentially authorized the software to scrape the data on their accounts. It's just a custom user agent running locally and the airline company has no business blocking anything.
In other words: "you can write this down by hand, copy paste or browser plugins but you cannot automate this". I wonder if this stood up in any other context and I can't imagine of a similar scenario from the top of my head where automation would be forbidden. I could totally hire a part time student from a developing country to do data entry for me and that would be alright? Strange world - somehow these corporations have people brainwashed.
The issue is not that some app has access to a timetable of work shifts. It is that it has access to credentials and potentially can so something else. In your analogy a part time student from a developing country data entry - this is scrapping public linkedin data. What happens here is an employee giving their office badge, so they can go get a folder from the employee’s desk, open it and make a presentation based on its content. To make it worse, many employees give their badges to the exactly same student.
Third party having an unrestricted access to the internal system. No sane business owner would be ok with it. This is literally the reason why protocols like oauth2 exists.
"One flight attendant said of the current situation affecting Sequence Decoder that they had “never seen a company go out of their way to make life harder for their workers.”"
Every large company I've worked for had an IT department (or similar) that intentionally made developers'/sysadmins' lives harder. They don't care. So the good people leave, morale drops, productivity drops, management starts implementing crap policies to force more work to get done.
Our IT security people give every impression that their preferred solution would be to drop all the computers in the river and lock the doors.
I naively went to them once with a suspected security intrusion event; they threatened to have me arrested. I couldn't convince them that I wasn't the one responsible.
At a previous company I worked, my team (system’s infrastructure) would purposefully avoid the cyber security team, and even attempt to sweep security issues under the rug, because they were so awful to work with.
I wish more people realized that HR, IT, and accounting are secondary services within a corporation. They aren't the core of the company. The people inventing, building, and selling widgets are the core. Everybody else is just a necessary evil that exists to help facilitate and streamline the stuff that matters.
Employees should not be bending over backwards to suit HR and IT workflows and policies. Policymakers should be bending over backwards to invent lean, effective processes.
The problem is that HR, IT, and accounting hold the financial and technical power. And of course their primary concern to use that power will be for the perpetuation and benefit of HR, IT, and accounting.
Same here. IT people would prefer to shut down the company rather than help it achieve its mission because they are heavily incentivized to prevent security incidents and not incentivized at all to help with the mission.
Couple that with a CIO who wants to build an empire and second- or third-rate MCSE "Certified" personnel and IT inevitably becomes a huge impedance against the corporate mission.
I think all of that comes down to incentive design being too narrow for some central departments. Terrible outcomes for the IT department are breaches and their own work going up a lot due to lower maintainability. Another example might be the legal department where the worst outcome is the company doing something that causes legal problems. Yet, the worst outcomes for the company of missing business opportunities because we ship too late or don't ship at all because we avoid all legal risk or everyone struggling with tooling.
Landgren would dig through endless convoluted menus to find out what his children were doing at school. If working out what his children needed in their gym kit was a hassle, then working out how to report them as sick was a nightmare.
This seems very removed from my own school days:
- find out what the children were doing at school: you couldn't, really, the curricula were not public. You could ask your kids after school, but I guess most parents were not really that interested anyway.
- what his children needed in their gym kit: shorts, t-shirt, sneakers. Every time. What else could it be?
- working out how to report them as sick: I suppose parents called the school in the morning(?) but as teachers were usually unaware of the reasons for a child's absence anyway they might as well not have done that at all. After the first term, the schoolmates would just inform all other teachers that X was sick, assuming so from the absence. Then, when X came back to school, they would bring a hand-written note from the parents explaining.
I truly think computerized school communication is worse than notes in backpacks and the occasional mailed letter.
They send way too much useless shit, way too often, so you end up ignoring a lot of it; use way too many channels to do it; design sites poorly—how about the year's calendar on the first page for a given school, above "the fold", with a link to subscribe your preferred calendaring software to it, right there, office contact info off to the side, also above "the fold"? And no horrible excessively-complex half-broken themes making it difficult to navigate? Nah, that'd be too helpful; and everything's in several different systems, all bad, and all in various states of up-to-date or neglected, so it can be hard to guess where you need to look for something and hard to know whether it's accurate when you do find it (especially if the apparent signal is "empty" or "nothing there" or "no information"—is that true, or has it just not been updated lately?)
The online gradebook was the "new thing" that I found to be a nightmare as a parent. The idea was that parents could log in to verify students are turning in their homework or get a heads up before the parent-teacher conferences if their child's grades are suffering.
The reality is that some teachers are good about getting assignments and test results input in a timely manner. Some wait and do weeks worth of assignments in one batch. Others input all of the term's assignments at the beginning and add results as they get turned in/graded, meaning student grades gradually move from failing to the final earned grade once the last exam result is added. The grades shown on the online portal had nearly nothing to do with the reality in the classroom and ended up being counterproductive. I'm glad I no longer have to deal with that system.
Worse yet, many parents of college age students expect their university to offer this same portal, despite the fact that it would be illegal for schools to do so without written permission from the student. It's been eye-opening to see parents on one message board I follow furious that they can't know their student's grades up to the minute. I can't imagine how awful those parent-child relationships must be.
I fully agree, it is most expensive, less clear and sort of makes it less personal. And a nightmare for privacy. A whatsapp group chat for public announcement and phone calls/texts/meeting with the teacher proves to be more efficient.
Swimming trunks/swimsuit. Clean indoor trainers for basketball/volleyball/circuit training/etc, or scruffy outdoor ones for cross-country running, orienteering or anything muddy outside. In winter they may need to take something warmer if there's outdoor activities planned, some locations in the north or near mountains may even do skiing or cross-country skiing. In our school in winter time we sometimes did scottish country dancing, so you didn't really need "normal" P.E. class gear if that's what was on.
I don't know what this school in particular offered but there's a bunch of possibilities.
Ah come on "multiple workout outfits" is a little bit of a stretch of what I said. Everything I described except for the skiing stuff is normal in anything but the very poorest countries - in addition to what you listed, you didn't have a jumper for when it's cold or some old trainers for when it's muddy? I just wore my normal shorts for football whenever we went swimming (fyi my school didn't have a pool, but some do) but obviously that's not practical for girls so swimsuit was worth mentioning.
I never got to go skiing as a kid, but in Central Europe (Czechia for sure, but probably Austria, Switzerland and Slovakia) and the Nordics (importantly where that story took place) it isn't just a fancy pursuit for rich people. I know people earning like 20000 CZK/month ($10k/yr) here who go skiing.
To be fair, I had everything I wanted, which was food, shelter, access to a powerful personal computer, and a lot of old sci-fi books. If other kids had all been equipped like you describe and I'd asked, my parents probably could have scrounged that stuff up for me.
I just wore my regular clothes to class and gym class, although in winter I'd change into shorts instead of exercising in jeans. I had a coat that I wore on the walk to school and one pair of shoes. Sometimes I'd have boots, but my family could only afford/only knew about cheap crap boots that'd disintegrate pretty quickly.
It blows my mind again and again living in the SF Bay Area nowadays, watching a brand new BMW pull up to Starbucks and unload a bunch of teenagers who proceed to buy $40 worth of coffee and snacks. Those dang kids don't know how good they got it! I grew up near the poverty line in the USA, so I can only imagine what it's like for engineers who grew up in actual poverty in other parts of the world.
I do not think having multiple pairs of gym shoes is normal outside of the very poorest countries. I grew up in the US and I never had multiple pairs of gym shoes. Neither did many of my friends. We had "shoes" and "dress shoes" if we were fortunate.
Cross-country skiing is cheap and affordable for nearly anyone. All you need is a pair of used skis, a pair of boots, and the ability to bum a ride from a friend.
Well it's a bougie hobby if you fly every year with your private school to Switzerland, and get a new set of skis (deliberately exaggerating of course, it can be various levels of bougie). Cross-country is definitely cheaper (no ski pass, skis are generally cheaper) but there are plenty of places where affordable ski slopes are nearby and it's normal for someone working in (for example) a pub to own a set of skis, and be able to head out with some friends for a day of skiing
I grew up in the USA and only ever owned two pairs of shoes at once: my regular shoes, and shoes for church that were several sizes too big so they wouldn’t need to be replaced.
I grew up in a middle-class family, and as a kid I was generally expected to be aware of when we had a pool day, or a skating rink day coming up, and to be responsible for bringing the right clothing/sundries on every day in school. Using the analog technology of 'remember it', assisted by 'write it down in your day planner'[1].
[1] Which I would never do. My memory, as an eight-year old was, of course, infallible.
I'm not sure how it is in Stockholm, but in Norway sometimes we had gym outside in a park instead of inside, so different shoes and clothes. And if there is a school-outing to a museum or a montain hike or the dreaded "skidag", skiday, where the entire school would travel up into the montains and do various snow and ski related activates. So it can be a big deal if you don't know what your kid needs any particular day. I'm born in 2002, so they usualy solved this with notes in our backpacks. I think they now mostly solwe this via SMSes to the parents, if it's not sensitive information.
Welcome to the new world. Teacher submits attendance on the computer at the beginning of every class period; if the student is absent and there's no excuse recorded then the system immediately starts robocalling all the family contacts.
For the public:
-Past and future departure information for all airports
-Past and future arrival information for all airports
-Individual flight info lookup
For flight attendants:
-Flight specific info inflight
-ETB alerts
-Open time alerts
-Flight alerts
-Open time display
-Sequence details
-Calendar
-Reserve call out list
-Standby list
-Crew chat
-Save notes about crew members
-Display prior sequences flown together with others
-Layover weather forecast
-Pilot sequence info
-Legality checker
-Hotel amenities and pickup locations
The critical bit is probably the "legality checker". Absolutely no way an employer would offer one of those.
Reading between the lines, flight attendant scheduling is probably quite .. adversarial, as shift scheduling at restaurants can be. The employer would like employees to have the minimum information so they meekly accept the shifts they're given.
Reading the descriptions under the sibling comment, it seems to be something that every airline absolutely should offer their employees. You don't want them to work illegally as that could mean huge fines for you.
Which clearly indicates that the fines need to me much higher, + probably specific accountability for individuals who made the calls, rather than just the corporation itself.
At most airlines there are two sets of scheduling rules. The regulatory rules, and the union contract negotiated rules. You can guess which one the company doesn’t want audited.
Violating "regulatory rules" should result in fines and the government coming down on you. I would think this is the worse outcome, but less likely than...
Violating "union contract negotiated rules" should result in grievances ending in legal action if the employer doesn't agree they broke the rules. This should be a lighter penalty but the union seems more likely to pursue the remedy than the government.
So which actually is it that the company considers worse?
Violating union contract rules requires the company to be called out on it. When the scheduling tools they provide to employees are archaic, strings of text spaghetti, and the contract is extremely complex, the likely hood of being called out on a violation by an employee is relatively low. Grievance settlements are often just a small amount of pay hours paid out for days off that are lost. Like maybe 4 hours of flight pay per day that there was an error.
I’m not suggesting they willfully violate the contract either. Many of the schedulers are simply ignorant of the nuances in the rules because they’re complex. They are just trying to get trips covered. When employees are armed with knowledge greater than the scheduler(tools such as this, or often just experience) you end up with phone arguments/discussions that cause delays and cancellations. The company would far rather that they keep moving the jets and settle grievances for the handful of people that catch issues after the fact.
>>This kind of old, hierarchical thinking from the 1950s needs to go away.
Yes, it does, but no, it won't.
Just as democracies must always be better armed and prepared than expansionist autocracies, or the autocracies will take over, people need to be better armed and prepared than those who want to exploit them, or they will get exploited.
Unfortunately, the exploiters often arrive better armed and prepared, and it is an uphill battle.
Often, the only winning move is to not play the game.
Well the winning move in these cases is usually unionization - when a large number of employees are being exploited by an employer a union that allows collective bargaining can force the employer to act in a more sane manner.
> Washington, D.C. (October 4, 2022) — The Federal Aviation Administration (FAA) today finalized a rule requiring 10 hours minimum, non-reducible rest for Flight Attendants between duty days – finally implementing a 2018 law that will make aviation safer for over 100,000 Flight Attendants and the passengers in our care. The rule will increase the rest period to 10 irreducible hours when scheduled for a duty period of 14 hours or less. Sara Nelson, president of the Association of Flight Attendants-CWA, issued the following statement ...
What would one have in a contract to help this issue?
A union is indeed a good form of being forearmed, and more should take advantage of it.
That said, it does not prevent all exploitation. IIRC, the FA's in this story have a union and a contract, yet this form of exploitative behavior is not prevented.
It should be the job of the employer to ensure scheduling is in compliance with safety regulation and agreements negotiated with the unions. The app should be necessary, but given that it is, that can easily be viewed as American Airlines knowingly breaks rules. That seems legally questionable.
My guess is that it has something to do with whether or not a trip can be legally flown by an airline employee given the amount of hours of flying time they already have clocked for a given period.
These aren't labour laws strictly speaking, more like airline safety laws. I.e. they aren't about well-being of employees but safety of planes operated by overworked crews.
> Although the law was passed back in October 2018 and was meant to be implemented within 30 days, the Trump administration allegedly put the measure on the backburner and “on a regulatory road to kill it”.
> The previous administration initially blamed the delay on a massive backlog facing the FAA caused by a partial government shutdown and then the worldwide grounding of the Boeing 737MAX.
I would imagine that pretty much every airline offers some sort of legality checker to their crew members. If an FA worked a trip illegally that would mean heavy fines from the FAA and from whatever union contract they also violated.
They don’t. Employees are responsible for checking the math of the scheduler. in the case of pilots, the employee has liability in this situation as well as the airline. Especially in situations where there is bad weather nationwide, employees can be and absolutely are scheduled incorrectly by schedulers and employees have to maintain several sets of numbers (rest time, etc) besides the flying hours that are logged in the airline’s app.
How does the FA know to complain without a legality checker? They have to go through the work of parsing the legalese and calculating time zone differences etc. And if they did complain, the airline says "Sorry, here's a freebie for your trouble" (worth 0.01% of the fine)
Compare that to every FA knowing every single time there's a breach, being able to compare notes etc.
2. How is their employer not providing this functionality?
I'll tell you how. It's because the app doesn't exist under the regime of a CIO and multiple layers of project managers, program managers, extensive regulatory requirements, fiefdoms, competing internal political interests, and the other various "enterprisy" crap that will kill off any entrepreneurial spirit.
They are, it's just spread across multiple websites and datasources as is typical in large corporations. This app scrapes all the sites with the FAs AA credentials and aggregates the data.
My wife and I are going to be doing the whole digital nomad thing across the US, Canada and Mexico starting in less than 2 weeks, getting one way flights and staying in hotels.
Currently, we have been flying out of a major Delta hub and we took for granted what an airline app should be.
Now as we are planning flights between Delta, American and United, I can confidently say that American has by far the worse consumer app of any of the major airlines or hotels.
I wouldn’t be surprised that their internal apps and websites are a similar shit show.
I'll take a flight on Delta with a layover over a flight on American that's nonstop. Of the majors, it's far and away the best. And their app is very good.
Well, I’m a citizen in the US so that’s not a problem and where I will be spending the vast majority of my time. I’m scheduled to be in Mexico at most 35 days and probably won’t be working at all in Canada for the 10 days I will be there.
I’m on a lot of calls and travel within the US for work occasionally. While my company will fly me from anywhere and to anywhere in the continental US, I can’t ask them to fly me internationally
Travel outside of the US is mostly vacations and work just enough days so I don’t burn through my vacation days
Working remotely for your employer (as in, you're not traveling for work, but working while traveling) is not in breach of any tourist visa restrictions.
In reality, you are basically relying on the fact that no one can tell when you open your laptop whether you check personal email or work email etc. But legislation wise you very well might be in breach of visitor conditions. And if you stay long enough, taxes might kick in.
Canada is a notable exemption because IRCC issued a note on What kind of activities are not considered to be “work”?:
> long distance (by telephone or internet) work done by a temporary resident whose employer is outside Canada and who is remunerated from outside Canada;
However, unless the relevant immigrant authority or legislation did this , the law still applies.
A C-level executive at a place I worked went to another country specific to scout out talent the company could hire remotely. Given that he was obviously working there I asked how he got a permit for that. His answer was that he got a tourist visa and was on vacation, so it was obviously legal for him to work while in the country.
> His answer was that he got a tourist visa and was on vacation, so it was obviously legal for him to work while in the country.
Well, that depends. For example, the United States has a B1/B2 visa where you can do business things like meetings. That's not work. But still, if you are a tourist aka you get a rare B2 only visa then no what he did was not legal -- and just because he is C level doesn't mean he won't breach the law in this. Might be out of arrogance or ignorance, even.
His answer sounds like "I am a rich man, laws, pfft" to me.
I don't think it was quite that. But from the conversation, he apparently was of the opinion that wherever the business was incorporated at was his place of work. As in even if he was in Norway but the business was registered in New York, he'd be working in New York.
I'm not an employment lawyer, but I don't think it works like that.
THIS IS FALSE ALMOST EVERYWHERE IN THE WORLD AND IF DISCOVERED, CAN RESULT IN CRIMINAL AND/OR TAX PENALTIES.
In general, it’s not legal to be an unregistered digital nomad unless you have the right to work in the country you are in. If you enter a country on a tourist visa, the primary purpose of the trip must be tourism. If you enter on a tourist visa for the intention of being a digital nomad, you have violated the visa.
In practice, it probably doesn’t matter as long as you are discrete. it's difficult for governments to detect this practice and many countries even tolerate the practice. But since being a digital nomad is almost always technically illegal, travelers have had challenges when they are discovered by the wrong person.
The bigger concern is usually your employer. Working overseas without approval is usually a friable offense.
When Canadians travel down to the US for work, they are always instructed to answer any questions with 'I'm just here for meetings/training', because telling CBP that you're going to be doing any programming for your Canadian employer while in the US is a great way to get denied entry.
I'm not sure how this works in the reverse direction, or in Mexico.
At least for driving, into Mexico, there is no real check. For anything. You merely drive through at speed while border guards glance into the cars. You don't actually speak to anyone unless they flag you. I think the U.S. is the only one of the three that would really care.
Canada definitely cares. Driving into Canada, every vehicle has to stop for an interview. If anything is off, they will send you off to the side for more intense scrutiny.
Ah sorry, I just meant about nitpicking the "working while traveling thing" -- but I was just guessing. Seems like they might care more than I expected.
This is notably distinct from visa requirements, but taxes are typically owed in the jurisdiction in which the work takes place, if you are there for a certain period of time or longer. The exact time period varies based on locality (state/county/city in the US).
It is complex enough that the only answer that is definitely wrong is "I'm just traveling for me so I can do whatever I want."
> Kansas has policies that could deter remote work and are deterring greater interstate work with our neighbors. Kansas requires employer withholding for people working in the state just for one day, which creates an annoying hurdle for companies trying to operate even in a small capacity in Kansas. In 2020, the Kansas legislature considered a bill that would have extended the withholding requirement period to 30 days, but the proposal died
I assumed this was always the case, just that no one except professional athletes actually do it. The CA FTB makes me afraid to take lengthy vacations in California.
> The term “domicile” has a special legal definition that is not the same as residence. While many states consider domicile and residence to be the same, California makes a distinction and views them as two separate concepts, even though they may often overlap. For instance, you may be domiciled in California but not be a California resident or you may be domiciled in another state but be a California resident for income tax purposes.
> Domicile is defined for tax purposes as the place where you voluntarily establish yourself and family, not merely for a special or limited purpose, but with a present intention of making it your true, fixed, permanent home and principal establishment. It is the place where, whenever you are absent, you intend to return. The maintenance of a marital abode in California is a significant factor in establishing domicile in California.
As long as you're remaining transitory and not staying in a single, fixed, permanent home you should be fine.
Hiking the Pacific Crest Trail (takes about two to three months to get out of California) won't establish residency in California.
>California residency regulations treat W-2 work carried on in-state as California-source income. It doesn’t matter if the work is performed for a non-California business. So it’s fair to say that if the FTB audited a nonresident and found he was working remotely for an out-of-state enterprise while on vacation, the FTB would assess income taxes (though California doesn’t have a robust method for auditing this; it usually comes up, if at all, after a residency audit is already initiated for other reasons).
This is what worries me, say they start checking Air BnB contracts and start auditing vacationers.
If you’re a professional athlete, and I’m almost sure none of us are, you pay taxes for each state where you played a game. I’m not sure how it is apportioned.
American Airlines was featured in a video at Microsoft Ignite last week. They showed how everyone involved in a flight talks to each other through Teams and how Microsoft / AA has worked closely together to get this working. I'm wondering if AA doesn't want this app being used instead?
That's kind of interesting. I take so many flights I never really remember one from the next. But I do remember at least one flight attendant letting us know that videoconferencing in flight is a felony. I could never find any such regulation, but I guess that wasn't an AA flight.
How is the app getting the information for flight attendants, some of which looks like it might be security- and privacy-sensitive?
If it turns out that the app was getting confidential information using flight attendants' access credentials, potential problems:
(1) potentially leaking confidential information to other parties, such as app developer, partners, and other users; and
(2) potential additional weakness in handling access credentials.
If it turns out that special credentials weren't required for security- and privacy-sensitive data, then maybe that's a problem.
If it turns out that flight attendants were entering information considered security- and privacy-sensitive, then maybe that's a problem. (Though it looks like the article might've been prompted by the app developer, to pressure scraping access, so presumably there's something from scraping that the app wasn't getting or getting as well from user-entered info.)
The article mentions that it scrapes data from the flight attendants' accounts. The data shown is broken up between "for the public" and "for flight attendants".
It would be surprising if AA doesn't offer this set of information already to their employees via internal systems.
It's difficult to see why AA would allegedly want to block it since it seems like a net benefit for their employee's productivity.
Maybe AA IT hasn't yet combined all this into one big user-friendly app, and therefore they want to punish the perceived disloyalty of the flight attendants.
It seems pretty clear that they want to block it because they want to control how the productivity information is accessed, and that control is more important to them than their employees being satisfied and productive. The article mentioned that the author of the app even previously reached out to partner with them (maybe asked for an API so he didn't have to scrape?) and they were still uninterested.
Imagine your company's official expense reporting system is a 35 year old Windows 3.1 application that constantly crashes and doesn't even integrate with payroll so underpaid assistants have to manually copy records over from DOS to ADP's system. One of those assistants comes up with a web-based app that scrapes the database and makes it easy for everyone. Most large companies I've seen are going to come down on that assistant like a ton of bricks, not reward them for their ingenuity. It's just the nature of power structure in large bureaucratic companies.
At a previous job at a certain 3 letter company, we once developed a simple web app so users could do the unimaginable: change their own passwords for an internal billing system.
Management congratulated us on the initiative and asked to shut it down immediately. The reason: the users had to previously open a ticket to get their password changed (very secure, right?) and, with this app, the customer would see a 30% reduction in tickets and would probably reevaluate how many bodies they rented from the 3 letter company.
So yes, I fully agree that most large companies are not interested in helping their users or workers at all.
The article is annoyingly vague about what information the app actually accesses/provides, so we kind of have to guess and assume it's some kind of information that makes the employees' jobs easier.
It's not (necessarily) gossip, if that's what you mean.
Flight crews are generally only together for a couple of days at most, which means a lot of the time you go to work and have a whole set of new coworkers you've never met before - or wait, maybe you did? Like, last year? Do you remember him because he was a ton of fun or because he was an asshole? Should have written it down.
You’d be amazed the creative places flight crew find to write down names/notes of people that are just truly wonderful to work with. Each fleet generally has a known place to look and see some notes.
On the developer's Patreon (https://www.patreon.com/tschaff) there's a few more details. He's a former AA flight attendant, seems like there was a somewhat similar project underway internally in the airline union which he was on but got kicked off of and built this separately.
We have bots scraping some of our pages at work. We've attempted to reach out but haven't received a response. We don't mind the bots so much themselves, but we want them to be well behaved. Currently they are making calls over and over again that return a 4xx response and are a significant portion of our traffic. We want to request that they stop making bad requests and slow down (we do have throttling in place, but this just gave them more errors to ignore and retry.).
I'd love for an open third-party like this one. It'd even help with prioritizing features that we're missing in our first-party products.
> but this just gave them more errors to ignore and retry.
So null-route the offending IPs on a [0]24-hour timeout? The problem you're describing isn't "scraping", it's "low-grade denial-of-service attack (that you suspect might be a result of attempted scraping)", and should be addressed accordingly. (The parenthesised part doesn't really matter.)
0: exponentially increasing up to -, for automated versions, but you're presumably already familiar with the current batch of offending source addresses.
Also, double check that your first-stage throttling actually increases the latency of the requests, such that a user-agent that doesn't issue multiple requests concurrently (but starts a new request immediately on recieving a response) will automatically self-rate-limit. This should be standard for any 'serious' HTTP server, but I've seen a few that incorrectly go straight from "serve 200 OK instantly" to "serve 429 Too Many Requests, also instantly" rather than "serve 200 OK after ~1 second", and sending 429 only when there are actually too many requests (in particular, more than one at any given time).
5.2.2 Third-Party Sites/Services: If your app uses, accesses, monetizes access to, or displays content from a third-party service, ensure that you are specifically permitted to do so under the service’s terms of use. Authorization must be provided upon request.
Hmm honestly it looks like they'd be in the right to take it down too. I support the app and I think it's cool what they're(the app makers) doing, but it's not surprise that app stores might not be cool with the DIY nature of how they get the data.
if it's not illegal to acquire the data, then this is the app store likely getting a call from AA and shutting you down. meanwhile, if you're this Jeff guy, you can submit a request to apple or google and get an email response days/weeks later. what's cool about that?
Everyone has “that story” about one particular airline but AA is particularly bad because a large portion of their fleet is old and requires a lot of maintenance which unsurprisingly causes delays and cancelations.
I echo the statement that I’ve never had an uneventful flight on AA. I fly a lot and can confidently say other major airlines are more reliable but usually also more expensive.
Pre-bankruptcy AA was so good. Exec plats were treated super well and their fleet was massive— even if a plane had an issue they could almost always hot-swap it out for another one.
Despite the website feeling dated now it used to be one of the only airlines where the website accurately reflected the backend system (even phone agents couldn’t touch your booking if you had it open on the site) and their lounges were solid too.
These days I agree they’re towards the bottom of the stack sadly… everyone else caught up and they stayed still.
I mostly fly Delta now almost entirely for the on-time performance.
I mostly fly United due to living in one of their hubs and have become accustomed to reliable service on well maintained planes. Every time I hear complaints about rampant delays or cancelations, I wonder what year they’re living in. But then I end up flying American and I realize it’s not the year, it’s the airline. Even in their biggest hubs, there is always delay-inducing last minute maintenance with no hot swap available. I just experienced this recently and was immediately reminded of this time last year when they made headlines about canceling a massive percentage of flights daily for months.
I actually much prefer AA’s website to United’s. AA’s may seem dated but it’s fast and the UI predictable. United’s feels like it’s fighting me.
I am not convinced that AA is actively trying to block this app. I used to work for a major anti-bot vendor, so from that I know that all airlines suffer heavily from price scraping and try to protect their flight search endpoints. The impact to them is due to fees from the centralized data services that all airlines depend on.
"Never attribute to malice what can be explained by incompetence."
I fly American between JFK and SFO regularly, including today. They have the best prices these days for an "international" business class on this route. The food is good, the flight attendants are friendly and good at their jobs...
Automobile manufacturers offer pathetic "center console" software compared to Apple, Google. American's in-flight entertainment system makes auto makers look like geniuses. They just don't understand software.
Corollary: if you think that everyone is incompetent, perhaps you're missing something.
For instance, you fly with AAL because of price, food, service. What sort of entertainment system would change your mind? It is hard to create something that works for everyone, is integrated with a plane that doesn't have a lot of downtime, etc. I'd rather just use my own device, so the best entertainment system in the world is unlikely to change who I fly with. So why invest in that?
Or on the original topic: the problem is not that they can't develop a similar app -- if nothing else, they could license this one or just turn a blind eye. It seems like they don't want it to exist.
It's really not. From what I've seen of crashes and boot up screens, most airlines seem to use some rebranded Android experience. I'm fairly confident that's what American used the most recent time I flew them based on some navigation buttons that appeared on the bottom at one point. There's a lot of Android devs out there in the world, but AA does seem to hire or subcontract to some particularly bad ones.
Then how do discount sites like Kayak, Expedia, or Matrix ITA/ Google Flights work without a hitch? The ticket prices are no lower than what SABRE will offer and it doesn't seem like AA is losing money in those cases.
Reminds me a bit of the dispute between McDonald's/Taylor and Kytch. An independent company Y stepping in to fix a particularly broken aspect of another company X's workflow (or literally a piece of hardware), company X responds badly, company X doesn't have the internal resources to do better, or company X is purposely not fixing things for some dumb and/or profit-motivated reason.
A possible solution: build your scraper as a Chrome extension and have legitimately logged-in users periodically hit that SCRAPE button.
AA sounds like a horrible company. They had a design legacy from when their logo and identity were designed by Massimo Vignelli, but they threw it away and didn't seem to have any concept of its value. Their website has terrible design and usability. When a blogger posted an article about how bad it was, one of the designers at AA reached out anonymously and said they understood his criticisms and that they were working to make the site better. AA searched their mail servers to find that employee and fired them for... caring, I guess?
I came here to articulate the counter position to this because people are solidly on the side of the app developer. If you run a business and your employees access your information to do their work through an un-contracted intermediary, it's a real security risk. The information can be filtered or maliciously corrupted, and the company can be easily blackmailed.
I don't think the law offers any direct remedy. The best thing for AA to do is force the app developer into court and make them pay legal fees if they don't want to contract with them, compete by making something better for their employees, or contract with someone else to provide the service. It's costly on all sides but table stakes for running a business in a competitive market these days.
Are American Airlines workers unionized? I wonder if they could collaborate with their union and get additional protections to workers' ability to access this data portably.
Typical adversarial relationship that develops between companies and unionized employees. It becomes more about winning the power struggle than getting work done or having a good life. And also more about the rules in the current contract than anything else.
If the current contract doesn't have any rules about it, then employees are just stuck with it until the next contract negotiation between the union and the company.
"it displays information required by crew members to manage their rosters and work lives in a single app.
The app is particularly popular among the large number of ‘reserve’ flight attendants at American Airlines because it gives them more control over their schedules, and the app has other features such as a calculator to make sure crew are working to legal limits."
Yeah, this was what I was trying to understand. This makes it sound like the app could be a daily listing of flights they need to work. Is that all it is?
There is a top-level comment higher up that lists the individual features. One is notably absent from AA's internal systems: checking that FA schedules are consistent with the regulations governing how much they can work. But the crux of it is that is takes the FA's AA credentials, collects their personal data from disparate internal AA systems, and displays it in a sane way to the FA. And presumably at least a couple of the features (like the legality checker) are sugar on top.
I would not underestimate how terrible internal legacy systems at a company like AA could be. It seems completely reasonable to me that this app could save FAs an hour a day or more. And if they're like pilots (who are typically only paid on time from wheels up to touchdown) that is likely an unpaid hour.
There's been at least one incident where I spent hours sitting on a runway after landing. The pilot was professionally, but obviously very agitated. Does the crew not get paid for that time?
"Door closed to door open" is not just a figure of speech, so if the doors were still closed and the plane was not yet at the gate: yes, they did (thankfully).
My source here is that my wife was an FA for many years. She has a couple stories of pilots deliberately not pulling all the way up to the gate if they know they're going to have to wait anyway, purely so the crew can stay on the clock (and it doesn't make any difference to the passengers either way). Though for every one of those instances, there are a hundred of being delayed in the airport or after-boarding-but-before-closed-doors or whatever that ends in hours of unpaid work time.
As someone who is currently consulting on an airline industry project, you are absolutely correct in identifying how terrible the internal legacy systems are. Everything is built on SABRE, which was first created in 1960 and absolutely shows its age. If SABRE was a person, it could retire.
What's worse though is that inside the industry there's little awareness of the possibility of doing things differently. Everything is just layers of wrapping around the old system without any abstraction of the processes.
The article suggests this is a single app which offers a single entry point into many disparate systems, which currently flight attendants have to access individually in order to do their jobs.
Past and future departure information for all airports, Past and future arrival information for all airports, Individual flight info lookup
For flight attendants:
Flight specific info inflight, ETB alerts, Open time alerts, Flight alerts, Open time display, Sequence details, Calendar, Reserve call out list, Standby list, Crew chat, Save notes about crew members, Display prior sequences flown together with others, Layover weather forecast, Pilot sequence info, Legality checker, Hotel amenities and pickup locations
"it displays information required by crew members to manage their rosters and work lives"
What does this mean? I read the entire article and still have no clue what a "roster" is, why a flight attendant would need to "manage" such a thing, and what data is being pulled to make this easier/better
"‘reserve’ flight attendants "
Also have no idea what a "reserve" flight attendant is...
Edit: Apparently there are some of you you think I actually need these words defined. I do not. ¯\_(ツ)_/¯
My second-hand impression is that aircrew scheduling is a bit different from most jobs.
The flight attendants are not assigned a fixed schedule, but instead “bid” on specific trips: depending on their preferences, some crew might want flights of a particular length or on a certain type of plane, to arrange layovers in a specific city (or none at all), or even to work with/avoid certain colleagues.
They also alternate between periods where their bids set the schedule (“line”) and where they are on reserve/standby to fill in on flights where someone is missing. This could be because of illness, but flight delays and working hour restrictions (via safety regulations) add a lot of complexity. Another added wrinkle is that most crew don’t get paid until the flight doors are closed, so they really do not want to spend a lot of unnecessary time at the airport.
Thus, it’s a lot worse than managing schedules at a coffee shop, and I can totally imagine how a specialized app could help.
Many flight attendants don't have a regular schedule. Instead, they can browse upcoming flight itineraries and choose ones they are interested in. For example, a sample itinerary might be Chicago to Cleveland to Boston to New York to Chicago over a 2 day period.
All of these features are available in the internal applications provided by the airlines, but they're not always that easy to use or flexible. Obviously, as a flight attendant, being able to get notified of new itineraries, book quickly, and perform complex searches is a big advantage to getting better flights, so there is a small but very motivated market for custom apps like these.
Of course, these apps aren't supported by the airlines, so they operate in a grey area and use techniques like credential passing and screen scraping to get access to sensitive internal data.
These words aren't specific to flight attendants. They're just english words with standard definitions. You're reading this on a device with an internal dictionary as well as internet access.
I didn't write the article, but I wasn't aware there were English speakers who didn't know the word roster either. It's a calendar of times employees have to work i.e. It tells you when your shifts are.
What do you call that?
Also a reserve is something you have spare in case the original planned one can't operate. So a reserve flight attendant would be someone who is on call in case someone else is sick, etcetera.
Yes, “rota” would be the commonest word for this idea. Although I did immediately understand “roster” in the article, so maybe that’s also in use. “Rota” is definitely what I would say though
I worked briefly years back on a web app at AA that provided the scheduling features for flight staff that this app provides. I didn’t see what happened to it in the end, but reading the app’s feature list, it sounds like that app plus social elements and weather. From scraping, which is scary IMHO.
Is there a linked source or way to validate the claims? I'm curious to learn more but the author didn't link or reference any. The comments seem to verify it but I don't find them trustworthy either.
What seems unsaid in the article but seems to be the case, is that the app is logging in to American Airlines systems using the flight attendant credentials and then scraping.
If this is the case, I think American Airlines is justified in trying to block them. An app storing/using credentials for a different service is a data breach waiting to happen.
Capturing a user's credentials and logging in to a system they don't manage is nowhere near your analogy. Unless AA is operating a public API, this is an unauthorized use of their service, which they absolutely have the right to shut down and protect themselves against. If someone takes the key to my house and goes inside without my permission, it's still breaking and entering.
This would be like claiming you have to use a company-sponsored to-do list app, or a company-sponsored git client, or a company-sponsored text editor, or a company-sponsored FTP client.
I'd hate to be at any job that enforces any single one of those. There are many arguments against this app, but this is not one of them.
I'm not entirely sure that is a nonsensical argument.
I am permitted to use any client I like to perform SQL queries of our customer data, but if the client were to happen to route the data through a third party, I would be in employment-jeopardy breach of our security policies.
Similar rules goes for hardware: I can bring my own device for reading and locally storing our email and chats, but customer data is not to be accessed on any hardware not authorized by the company.
Roster data is not customer data, and there are reasonable arguments to be made that this is not an exact parallel. But in principle, I can understand a company wanting to have control over certain types of data and how it might be exfiltrated from the company, even if it is intended for employees to use to do their jobs.
But in this case, isn't the "third party" just a piece of client-side software that performs a bunch of http requests to systems the client user is allowed to access, on a device the user is allowed to use, in order to aggregate the results and show them all in one spot? It's not being sent to third party systems off the user's phone.
Banning it would be like restricting certain sql clients, like allowing the CLI clients, but banning pgAdmin or MYSQL Workbench.
I also don’t really agree with the ban, and seriously doubt that they have any reason other than, “We dunno what this is, and are too lazy^H^H^H^H busy to think it through, give a decision, and deal this the precedent of allowing screen scraping and/or third-party clients.”
All I was trying to say is that while I may disagree with their call, I wouldn’t go so far as to say it’s “nonsense.” Just wrong :-)
Really? Maintaining consistency across users is a huge part of IT. I can't imagine working somewhere where everyone just uses whatever software they want. Support would be a nightmare.
Any evidence that the app is storing/using credentials, or if it is, storing them insecurely? How about AA's own apps? Do they store credentials or do you have to log in every time?
There are no comments/voice on the AA side because they are likely 'accidentally' dodging labor laws (by laying obligation of checking legality on the attendants) through obfuscation of this data. That seems a large part of why it's become a 'must-have.' That kind of behavior merits this relatively 'soft' one-sided take in my opinion.
Then perhaps AA corporate should have responded more promptly to the writer’s request for information had they been interested in being more fairly represented.
By that reasoning, if the company refuses to comment an article should not be published, or if published must be rejected as one sided? That's convenient.
You can always still publish when they don't react within a reasonable time, and add a comment to that effect, as you can often see in newspaper articles.
It is commonsense to use the hardware/software authorized by your employer to do the work.
One side story. It is third party app that is not authorized by AA. Instead of get permission through a contract, the app developer scraped AA's data without AA's permission.
Nope. You don't need permission to scrape publicly accessible data, and that's only on the legal side of things. Data behind any kind of login is not fair game. Apple also has rules against use of any scraped data you aren't explicitly authorized to access.
Apple is of course fine to prohibit whatever they desire for appearing in their store, but scrapping is a different issue.
Copyright etc could restrict copying information displayed on a website, but if someone can legally write down information via pen and paper they can see then they can scrape it. The process being automation doesn’t inherently matter.
App screenshot on the appstore shows a list of names associated with a rota, presumably crew member names scraped from the airlines' password-protected crew portal.
It's not especially surprising airlines don't want unauthorised third party apps accessing and storing personal data from their intranets, even if the third party developer is very ethical about not leaking it to people without passwords and makes beautiful UX
They don't have a right to scrape password protected personal data off an intranet, and AA also have the right to attempt to block an app or its scrapers regardless of whether the scraping is legal or not.
In this context blocking scrappers is probably legal for AA. However it might be illegal in other contexts, such as if there are any concerns around disability.
All the rulings here have enforced employers rights to their data. The google engineer using their login to scrape users private emails and then running it through a third party platform would be fired.
While true, is that relevant? This app is free and doesn't even have in-app purchases, so it sounds like it was just built by a person (one sufficiently human to its users that the reviews are directly written to or about "Jeff") who had some reason to care for this community and isn't a "business".
While true, is that relevant? I can easily see Apple siding with a business as large as AA over a single person trying to make a useful app for people.
This site is called Hacker News. This is an exemplary application of the hacker ethos - to apply skill and insight to make a digital system better for some group who is being underserved, without all-encompassing regard for the desires of the offending organization.
Well, it's not a business, and it doesn't adversely affect the other business so this is fine. The complaint isn't really by the app developer. It's mostly that life is being made hard for the attendants through blocking the developer.
this may be true, but it's very disappointing. As BigCos keep getting bigger and owning more and more data, it is essentially a moratorium on individuals implementing good ideas that provide value to other individuals.
As citizens we should oppose this concept of data ownership; you shouldn't be able to 'own' facts that the public and your employees already reasonably have access to.
Rule #1-1 doesn't really apply though. American Airlines seems to be shooting itself in the foot by making working for them harder. This is somebody doing something for free that brings them value.
Yet another reason to build a website instead of an "App".
It seems to me that all this could be delivered via website or PWA... Then no one could block it.
Careful client side caching would help for times of no connectivity.
I'm a big advocate for PWAs, but this is a backend data issue. American Airlines isn't forcing app stores to take down the app, they're adding anti-scraping measures to prevent the app's backend from being able to request data for flights, shifts, etc.
This particular case is a bit harder since it's not purely using public data, but may still qualify since it's likely scraping with legally-obtained credentials.
I know of businesses (scraping for ride-sharing, scraping for business intelligence for retailers, scraping from LinkedIn - see HiQ Labs v. LinkedIn) that have continuously succeeded via scraping in ways that large businesses oppose.
The key is: you must make enough profit to justify dedicating engineering and legal techniques to defend your scraping.
- Scraping public data is legal, as affirmed by the Supreme Court in Van Buren v. United States [1] and HiQ Labs v. LinkedIn [2]. Defending yourself or suing the data owner in court are both expensive though
- Defeating anti-scraping via technical means is pretty much always possible, but can be costly depending on the scraped site's technical expertise and value in keeping their data private. The benefit to you must exceed the cost to you, and ideally should also exceed the cost to the data owner
- Mobilizing PR and internal resistance may also be effective, but it's usually hard to have outcry from a large enough group to change an organization's policies. In this case, the union can push for it, but AA may try to withhold improvements until the next set of union negotiations
1. https://en.wikipedia.org/wiki/Van_Buren_v._United_States
2. https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn