I recently hired into a data analytics team for a hospital, and we don’t have a style guide. Lots of frustration from folks working with legacy code…I thought putting together a style guide would help folks working with code they didn’t write, starting with requiring a header for SQL scripts first as low hanging fruit.
Or so I thought.
My counterpart over application development says that we shouldnt be documenting any metadata in-line, and he’d rather implement “docfx” if we want to improve code metadata and documentation. I’m terrified of half-implementing yet another application to further muddy the waters–i’m concerned it will become just one-more place to look while troubleshooting something.
Am I going crazy? I thought code headers were an industry standard, and in-line comments are regarded as practically necessary when working with a larger team…
I recently hired into a data analytics team
I work in Data Engineering and have spent most my time on analytics teams. They don’t have a SWE/CS background and generally because of that don’t follow any good programming practices. In my experience style guides are hard to get them to follow properly even if you set up SQLFluff for them., I can barely make them see the advantage of not committing directly to main (at least we’re using git). It’s very frustrating.
Yep that’s us–maybe half of us have CS degrees.
The funny thing is that the pushback is coming from the “regular” development folks. At least we’re using git too :)
I think it depends on what you want to put in the header. An overall description of the file, it’s goal, etc - that sounds great.
A changelog saying who changed what, when? Our version control system is tracking that for us already.
Hmm, do I want to open some external site/program to see my documentation or have it already in the code in front of me?
We use doxygen at my company and I think I’ve only ever opened it twice in 9 years.
Doxygen may be required in regulated industries like healthcare, banking, or robotics, but programmers never use it internally. The headers themselves are useful though and show that programmers take care of what they write even if they don’t read the generated HTML.
Commenting your code is an industry standard. Some kind of separate comment metadata won’t generally work, unless you code exclusively in spreadsheets, where you can add a note to each cell.
Given that it’s a hospital, I guess your management will listen to whomever performs bureaucracy better, not whomever codes better.
Yes, serious people write docs. I hate this bullshit about code that should be so good that it’s “auto-documenting.” It never happens in real life. Code is at best of average quality, but it needs documentation. At my previous job they had “guidelines” to make sure that code didn’t needed doc. It was a bad joke and we had the worst code I’ve ever seen.
I don’t have solutions for you though. You need a combo of documentation generation, code formatter (in the CI maybe, or before a commit), and code linters to check for errors.
Writing self documenting code reduces the need for comments significantly, but you’ll still need to write docs and even code comments when needed.
I had a lead architect at one of my previous workplaces who outright forbid writing comments, otherwise the build would fail. That lead to convoluted and slow solutions in order to make the code readable, or just parts that nobody wanted to touch because nobody understood them.
My point is that you should strive towards self documenting code as much as it makes sense, but don’t take it to mean that you should never write comments.
People should be able to tell what your code does without going deep into implementation details but that’s not always possible, especially if you’re working with lower level languages with fewer abstractions, or projects with complex algorithms or architecture.
Its wild to me that some people hear “your code should be self documenting” and take that to mean “never write comments”.
All self documenting should mean is I can look at a method and get a general understanding of what it does, and it shouldn’t have any unknown functionality. Specific implementations, design quirks, choices that might only make sense if you know business context should all be comments in your code.
On the other side of all that I worked with someone who insisted methods were documented college style, the “authors” name, date it was written, what it does, why it’s here, our star sign. I hate that just as much, so much clutter.
Its wild to me that some people hear “your code should be self documenting” and take that to mean “never write comments”.
It’s the same people that hear “security by obscurity is not security” and take it to mean “publicity is more secure than obscurity”. A key being under a door mat is bad but putting it on top of the mat because “obscurity isn’t security” is silly.
Edit: A better example is having a hidden door with a look is still more secure than having a non-hidden door with a lock.
“Self-documenting” just means “(I thought) I understood it when I wrote it, so you should too”. In other words, it really means “I don’t want to document my code”
I like it better when the docs are embedded in the code or alongside them. Everywhere I’ve worked it is a pain trying to find some random Confluence page or whatever where some API doc is.
Also if it’s not in the code, it will get outdated quickly and nobody will ever look at it. Separate docs are only really useful for main concepts that are not going to change that quickly.
A header might be useful, although there’s likely better ways to (not) document what each sql statement does.
But inline documentation? I’d suggest trying to work around that. Here’s an explanation as to why: https://youtu.be/Bf7vDBBOBUA
If possible, and as much as possible, things should simply make enough sense to be self documenting. With only the high level concepts actually documented. Everything else is at risk to be outdated or worse, confuse
Here is an alternative Piped link(s): https://piped.video/Bf7vDBBOBUA
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source, check me out at GitHub.
What do you mean by Code headers?
I hope you don’t mean those “Created by:” and “Last edited:” things… If yes, please don’t!
Your friend may have a point.
It depends where the SQL is.
Is the SQL in a data model in an analytics platform? Some platforms will happily carry comments around like last week’s pizza during query generative phases of visualization, so it may not be appropriate to put comments inside a data model, as those comments could become bugs if the analytics platform is lame, like most are.
Others, certain flavors of SQL DDL (Tables, views, etc), comments outside the DDL don’t make it inside the resulting object, so headers may not be the right place either. Most RDBMS have meta-descriptors that can apply to DDL so those might be good to look at.
For arbitrary SQL, outside a brief inline comment describing why it exists, and what invokes it, your next best bet may be a link to a more descriptive data architecture diagram that shows how this unit of SQL integrates with others. You might prefer hyperlinked descriptions from that data architecture over searching thru code.
As long as comments don’t require continual parsing (a one-time tax is inconsequential), definitely add details that you have figured out so others don’t have to re-learn the tribal mysteries of long-deceased ancients.
