I literally told my boss that I was just going to rebuild the entire pipeline from the ground up when I took over the codebase. The legacy code is a massive pile of patchwork spaghetti that takes days just to track down where things are happening because someone, in their infinite wisdom, decided to just pass a dictionary around and add/remove shit from it so there is no actual way to find where or when anything is done.
I was generating a large fake dataset that had to make sense in certain ways. I created a neat thing in C# where you could index a hashmap by the type of model it stored, and it would give you the collection storing that data.
This made obtaining resources for generation trivial
However, it made figuring out the order i needed to generate things an effing nightmare
Of note, a lot of these resource “Pools” depended on other resource Pools, and often times, adding a new Pool dependency to a generator meant more time fiddling with the Pool standup code
Side-rant:
I rarely write Python code. One reason for that is the lack of type safety.
Whenever I’m automating something and try to use some 3rd party Python library, it feels like there’s a good 50/50 chance that front and center in its API is some method that takes a dict of strings. What the fuck. I feel like there’s perhaps also something of a cultural difference between users of scripting languages and those of backend languages.
What you described sounds so much worse though holy shit.
Yeah, the new pipeline is based HEAVILY on object inheritance and method/property calls so there is a paper trail for ALL of it. Also using Abstract Base Classes so future developers are forced to adhere to the architecture. It has to be in Python, but I am also trying to use the type hinting as much as humanly possible to force things into something resembling a typed codebase.
I literally told my boss that I was just going to rebuild the entire pipeline from the ground up when I took over the codebase. The legacy code is a massive pile of patchwork spaghetti that takes days just to track down where things are happening because someone, in their infinite wisdom, decided to just pass a dictionary around and add/remove shit from it so there is no actual way to find where or when anything is done.
FUCK. Triggers me. Just got let go from a place that had this problem and wouldn’t let me make any changes whatsoever. I didn’t even push hard.
I did this once
I was generating a large fake dataset that had to make sense in certain ways. I created a neat thing in C# where you could index a hashmap by the type of model it stored, and it would give you the collection storing that data.
This made obtaining resources for generation trivial
However, it made figuring out the order i needed to generate things an effing nightmare
Of note, a lot of these resource “Pools” depended on other resource Pools, and often times, adding a new Pool dependency to a generator meant more time fiddling with the Pool standup code
Side-rant:
I rarely write Python code. One reason for that is the lack of type safety.
Whenever I’m automating something and try to use some 3rd party Python library, it feels like there’s a good 50/50 chance that front and center in its API is some method that takes a dict of strings. What the fuck. I feel like there’s perhaps also something of a cultural difference between users of scripting languages and those of backend languages.
What you described sounds so much worse though holy shit.
Yeah, the new pipeline is based HEAVILY on object inheritance and method/property calls so there is a paper trail for ALL of it. Also using Abstract Base Classes so future developers are forced to adhere to the architecture. It has to be in Python, but I am also trying to use the type hinting as much as humanly possible to force things into something resembling a typed codebase.