Pyright is the Microsoft provided lsp: does anyone else feel uneasy about using this? Especially given the EEE playbook "that we definitely won't do this time" - would love to have a discussion on this if someone could talk me into using it.
I just use a combination of black, flake, and mypy. Apparently ruff is quite good as well.
For lsp I use Jedi, which actually worked faster on my company's code base than pyright, a year or so ago.
A slight correction: Pyright is the Microsoft provided _type checker_. Eric had no intention of ever extending it to a full LSP (though it is usually good enough) in fear of Microsoft abandoning it the moment it started to compete with Pylance [1]. Now that Eric has left Microsoft (but still is somehow the main contributor to Pyright) the situation is even stranger to me.
(In case you are wondering, I do not have the time to maintain an independent fork and I am not sure how Microsoft would react to one, now that I have heard Eric's concerns.)
This version of Microsoft has learnt from the mistakes of its predecessor, coming up with an even more potent model for making money.
"Embrace + extend + embed" makes more money than "embrace + extend + extinguish".
As long as Microsft controls the data (your vscode actions), they have no need to explicitly destroy their competitors. The data skew gives them a natural advantage. I don't trust their intentions, but I do trust the well-aligned monetary incentives.
Can the extension be used on VSCodium builds? If memory serves, C# tooling extensions are proprietary and are subject to unfortunate treatment when using open-source builds of VSCode.
Yea, and if you're a company making more than $1M in revenue or have over 250 PCs, then you need a commercial license.
Microsoft definitely seems to be pushing proprietary VSCode extensions where it's free to get users hooked on the product (and kill competitors) and then charge companies lots of money for the tools they got everyone hooked on because they appeared to be free.
I would not personally use any of them. Typed python in previous workplace was pretty much shitshow, and if starting from scratch instead of untyped codebase it might be worth it, maybe, for some specific cases, but in general combination of bad library typing and bad type checking tools leads to horrible user experience.
So going forward my personal projects continue to be ducktyped, and if I care about types, I use rust or go.
My experience couldn't be more different. I've had the experience of introducing type annotations into legacy Python 2 codebases and it's been a godsend and the only way to get them to Python 3. I've also had the experience of writing greenfield Python 3 code fully-typed from the start and its also been a useful tool there.
Yes. Mypy supports type annotations provided in comments in both Python 2 and Python 3.
I have ported a codebase from Python 2 to 3 by annotating it with comment-syntax annotations and it helps a ton to keep your strings and bytes separate. Basically, you annotate with the binary_type, text_type types the six package and typecheck your code on both Python 2 and Python 3.
Basically, the steps involved are:
1. set up static analysis (linting and typechecking) for python 2 and python 3
2. get code passing static analysis on Py2 and Py3
3. get code passing tests on Py2 and Py3
4. get code running in production on Py3
5. stop checking and testing on Py2
It's not trivial and it requires some strong Python experience to write code that correctly "straddles" Py2 and Py3. But the advantage of the static checkers is you can slowly dial up their strictness and once they are dialed in, you minimize new regressions (new untyped code, new code that fails typechecking).
Eclipse with PyDev had awesome support for types in python 2.* before they were a python 3 thing. It actually worked really well for the 90% usecase and was my secret weapon about a decade ago.
Whatever bad experience you get from the (admittedly) immature tooling, pales in comparison to the intense discomfort or working with an untyped Python code base of even moderate size, specially ones written by many different contributors.
Type-checking in general is good. However, Python's type-checking mechanisms are bad. The syntax for adding types is very obviously bolted on, they miss basic type errors all the time, and third-party libraries often miss type annotations anyway.
Python is fundamentally a duck-typed language, and that's how you should use it. If type-checking is important to you, use a language that is designed for it.
Eh it's either Python or C++ with ROS, and Python is much easier to work with. Adding type checks to method arguments isn't much different than doing it in Rust, and makes life much easier with anything larger a tiny program.
If there is anything complicated, you can either make a quick generic type (TypeVar) or even just throw in an Any type! I had to do that recently with a function that handled different ROS services, which apparently do not share any base class.
I have had the misfortune of having to work with untyped Python code. It is completely awful and I cannot imagine anyone genuinely prefers it. Impossible to navigate. Full of trivial type bugs and typos.
Doctor, it hurts when I punch myself in the face. I need something to prevent that.
I’m talking about realistic errors, not theoretical ones that would crash at first run. You have one test, right? Run the code once before deploying? I thought so. In which case, pyflakes and a few tests find a great majority (not all) of errors devs actually have.
> pyflakes and a few tests find a great majority (not all) of errors devs actually have.
No they don't. First of all people are really bad at writing tests. Secondly, writing tests sucks; I would much rather write static type annotations than tedious type checking tests: you know when to stop and you get the other benefits of static types like making the code easier to read and navigate.
Every piece of code I've added static typing to has revealed bugs. One we later hit in production (because the author was stupidly against static typing so I gave up trying to help him).
That bug was simply calling a method that didn't even exist. It was in a code path that wasn't tested, despite having some tests.
But I can see you're from the school of "I don't make mistakes" and convincing you otherwise is about as effective as trying to persuade Christians there isn't a god.
I’ve rescued two very large Python projects bordering on disaster, and have improved many more. In both cases the main problems were architecture-related, they were written at the wrong level of abstraction, then in the wrong language. Those need to be fixed first if you value your time. Typing errors were not a top-ten concern:
I.e. there are bugs, there are always bugs. But are they important bugs? Is the question.
When faced with a house of cards, do you fix the nails, or start with a new foundation?
Also you don’t write tests just for types. You write tests to validate functionality as you would anyway. And they will find type errors on failure.
Black/white adolescent appeals to the one-true-way are not compelling and why you did not convince, then or now. Typing is simply another useful tool in the toolbox, and I never said it should be avoided—my statements were qualified.
You're looking at a bridge made of wet cardboard and saying you shouldn't use FEA when designing bridges because the real problems are the fact that it's made of cardboard and it's wet.
> Also you don’t write tests just for types.
Not just. But you do have to write those tests as well if you don't use static typing. I have seen them written. Static typing reduces the number of tests you need to write. This is very very very well known.
> Black/white adolescent appeals to the one-true-way are not compelling and why you did not convince, then or now.
No, it's because you view static types as an admission that you might make mistakes and you don't want to admit that to yourself. You literally equated making typing mistakes with punching yourself in the face.
Pyright by default because it's a LSP so I get to go definition, find references, etc... as well as the type checking in neovim. It's Pylance as part of the Python plugin in VS Code. Just make sure you go to the settings in Vs code and turn the type checking on to basic or strict. (It's off by default)
I tend to run mypy every now and then ad-hoc to check things and it sometimes finds things that need correcting but I wouldn't say it's as worth setting up as the former.
I used Pycharm briefly in 2017 and thought wow this is great, does a lot of type checking just out of the box.
But after coming back to it in 2024, I found it quite lacking in multiple areas compared to LSP based type checkers. I work on a sizable Python codebase and opening it in Pycharm everything looks great at first glance. Then you open it in an LSP editor and you see warnings and type errors everywhere that Pycharm never caught and everyone using Pycharm never sees.
So, in my opinion, yes, much better than Pycharm, unless you have an affinity for Jetbrains products, LSP type checkers are much better. Again in my opinion, others may disagree.
The recent takeover of rye by the guys doing ruff[1] has some likelihood of rendering the python typing and packaging scene into something more coherent.
Having the tool get started with Ronacher, of Flask fame, gives it some mindshare among those paying attention.
It would be better still if the PSF were endorsing, but that gets too political, I suppose.
Would also be interested in what is friendly with Django. Using VSCode + the Python extension, the default checker (Pylance?) just pukes on the ORM magic.
Pyright is much much much better than Mypy, and it's easy to use with VSCode. The only reason to use Mypy is if you're adding types to a legacy codebase and don't want to be immediately confronted with a gazillion type errors, because Mypy has "eh whatever" modes and is generally more lax.
My experience is pretty negative with all of them, I would be almost more tempted to ignore type checking in Python until it has drastically improved. In particular, if you're experienced with Typescript, you're probably going to be very disappointed when trying to do similar things in Python.
The most recent one I've tried was Pyright. Pyright is faster than Mypy, and can also be configured to be quite strict, both big plus points for me. Unfortunately, in practice it ended up being far too strict in weird ways, and therefore difficult to work around with libraries that don't share its idiosyncratic assumptions.
For example, in Typescript there's an unknown type, which you can't do anything with unless you check its runtime type first. It's a clever way to add a true dynamic type to the type system without losing type safety. In Pyright there's a similar Unknown type, but in strict mode the type checker simply throws an error if anything is ever unknown. For me, this turned out to be particularly painful when using library types that involved generic state. Because I wasn't using that state, I didn't set a type for it, and it defaulted to Unknown. But that meant that an object had Unknown properties (that I was never using) which meant the type checker kept on failing. Moreover, because Python's generics story is pretty miserable, I couldn't figure out how to set the type to something not Unknown. And I couldn't figure out how to disable the error because Pyright's documentation is so poor. I genuinely spent a day of work on this problem, then disabled type checking completely because that was the easiest option.
I haven't used Mypy for over a year at this point, so things may well have changed, but my memory was that where Pyright is overly pedantic, Mypy was overly lax. It was also a lot slower, and had similarly poor documentation. The main reason I switched was that I was still regularly running into runtime type errors - the very errors that the type system was meant to help me avoid.
In both cases, I found the biggest problem was that neither tool seemed to go as far as Typescript in terms of modelling real-world Python code. What I mean by that is that idiomatic Typescript usually looks a lot like idiomatic Javascript in terms of the patterns and features used, because the Typescript team have put a lot of effort in to ensure that Typescript types represent how Javascript is actually used.
On the other hand idiomatic typed Python typically seems to look very different from idiomatic untyped Python, in large part because with types, you end up limited to about 1/4 of the surface area of the language. (This is why a lot of major libraries or tools come with type checker plugins because their library's behaviour cannot be described purely in terms of types.) Typed Python then ends up looking like a weird Java variant.
If I were going to add a type checker to my Python code again, I'd probably go for Mypy because it seems more like the standard and had marginally better documentation. But right now, I have the Pylance plugin installed in my IDE and that's about it - there's no type checking in CI, I just hope that I catch all the obvious cases with good enough testing. Thankfully the Python side of my work is pretty minimal, a few hundred lines of code at most. If I had a bigger Python project I might revisit this situation, but I'd be more tempted to just avoid Python for bigger projects.
i've often thought a more interesting approach for python's duck typing would be if the interpreter looked for anomalous types on assignment. so build a compact distribution of type "shape" for every assignment and then either throw exceptions or log warnings when apparent outliers are encountered.
That's roughly a structural type system, and it's part of what makes Typescript work well (although it has some disadvantages in that you can't easily describe the type "an object with only these attributes). You can somewhat get that effect with Protocols in Python, but it often feels a bit boilerplate-y to me.
1) i'm primarily interested in data types, rather than objects, as i believe most runtime errors are the result of unexpected changes to data types.
2) i'm suggesting a type system that does away with hand annotations and rather builds them on the fly by keeping assignment history and using that to determine when statistically anomalous types appear.
That would be a structural type system with Hindley-Milner type inference. A good example of that might be OCaml - it's largely structurally typed, and the types can be completely inferred without type annotations.
However, while Hindley-Milner will find all the type errors in your program, it won't necessarily report those errors in the most logical places. That's why it's generally still the norm to annotate functions with type annotations except in the most trivial cases: knowing that a function will take a string and return a number is useful information not just for the compiler, but also the programmer.
To come back to your first point, one common solution here is the "parse, don't validate" mantra for loading data. This is the idea that when accepting unknown data (e.g. from a file or API), you first parse the data and convert it into a representation that enforces the invariants your program believes about that data. For example, that names can never be longer than twenty characters, or that ages are always positive numbers. By doing this parsing as soon as the data enters your program, you immediately handle unexpected changes to the data formats - if data changes such that it can no longer be parsed, this is an immediate error, rather than something that will only show deep in your application.
With something like Pydantic (alternatives are available), you can describe what your data should look like in terms of types, and use Pydantic to do the parsing for you.
cool library. although worth keeping in mind that strong typing doesn't solve all your problems.
here's an example, taking the second example on this page https://docs.pydantic.dev/latest/ with the external_data dictionary, if i add a collision like:
'wine': 9,
'wine': 1,
it seems to just silently pick the last one, which seems error prone to me. maybe there's a flag to fix it, but i couldn't find any documentation or switches that controlled this behavior. :/
with respect to the approach i mentioned above, it's more of an idea (or probably a research topic) to do quantitative analysis on the behavior of programs while they execute, rather than static analysis of the code that defines them. the latter is obviously preferable, assuming the approach can capture all of the types of analyses of interest, the quantitative approach may be more suitable for large or legacy systems where detecting changes in data shape (even if of valid type) and branching behavior may be useful for tracking down causes of, or detecting conditions for, errant behaviors.
Pydantic operates on Python dictionaries, so questions about duplicate keys will come down to whatever the Python behaviour is. (I.e. if you have a dictionary literal with duplicate keys, Python will select which key "wins" before Pydantic gets a chance to intervene.) When loading directly from JSON, it's possible to hook into Python's JSON-parsing mechanism and throw on duplicate keys, which probably ought to be the default given the number of security risks that come from this case.
That was part of the wasted day I was talking about - trying to configure the system so that the rules would be more useful. I could disable the warnings per-line with comments - although this meant that my code was then filled with these lines and the formatting was awful - but I couldn't disable the warnings globally for some reason. I never managed to figure out why, unfortunately.
There is no perfect solution. However, I actually prefer Pycharm's built-in typechecker, because JetBrains is BY FAR the best at checking syntactically incorrect code, i.e. code that you're in the middle of typing out.
But, Pycharm's still quite limited compared to pyright or even mypy. So I find myself A) doing an extra linting pass with pyright or mypy; and/or B) adding redundant annotations/overloads to help pycharm figure out what's going on.
I use LazyVim (neovim) and VS Code for Python work. Pyright and Ruff work well with Neovim, and Pylance works only with VS Code (but you need to enable it, if I remember correctly!)
My only issue is I occasionally need to restart Pyright when it fizzes out.
My rule for these things is to stay close as possible to the mainstream, boring thing, closest to the Python Software Foundation, unless I have an overwhelming reason to switch.
The only useful types to add are Cython 3, otherwise adding more letters with no benefit reduces readability and reminds me of “public static void main”.
It's worth considering whether this is a helpful contribution to the debate. The OP obviously wants to use a type checker and thinks that's a good idea. If you believe there is a particular reason this is a bad idea, we might all learn something if you said why. If you think that type checkers are in themselves not a bad idea but all the existing python type checkers are inadequate or unhelpful, that would be good to know as well. Instead all we have is your snappy/snarky hot take and a bunch of questions.
I think you are reading far too much into their comment. I agree that their comment could have benefited from some elaboration but the simple act of them expressing their opinion (especially one that is not seen in the rest of the thread, as of my writing this comment) is a "helpful contribution to the debate", in my opinion.
Agree. It also seems a direct contravention of the guidance "Please don't post shallow dismissals.... A good critical comment teaches us something. " https://news.ycombinator.com/newsguidelines.html
I didn't know about the guideline, so thank you, even if I disagree with it. A shallow dismissal is anyway a signal you can ignore, but meaningful, especially when people flock and say the opposite.
Maybe people don't have definite ideas about things, and just want to say "no" because they feel like, and that's freedom, imho.
Anyway I don't use types or type checking, I think that they are the crippling reason why I switched to Python in the first place. There are plenty of typed languages or IDEs that do completion nowadays (which I don't use), but the freedom to completely rearrange the architecture of a program that comes from very loose coupling is not easily traded for compilation time warnings. We've been doing tests because typing blocked us from development, and we just wanted to be able to screw things if we wanted.
The company I work for, Odoo, big opensource ERP, also doesn't have types at all.
If I wanted types, or a fast language, or "precise interfaces" I would definately rewrite. When I developed in C#, most of the coding time was spent on typing, interfaces, adapters, factories.
I just use a combination of black, flake, and mypy. Apparently ruff is quite good as well. For lsp I use Jedi, which actually worked faster on my company's code base than pyright, a year or so ago.