Like what you see? Subscribe here and get it every week in your inbox!
Issue #251 - December 31, 2023
Here are the top threads of the week, happy reading!
1. Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?
Top comment by ilaksh
You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for.
You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG.
Test it out. If it really and truly doesn't work, search for a script that creates question and answer pairs automatically with gpt-4. Then try using that for qLoRA. I have never heard of anyone successfully using that for a private document knowledgebase though. Only for skills like math, reasoning, Python, etc. I think the issue is that you need a LOT of data and it needs to repeat concepts or any facts you need to learn many, many times in different supporting ways.
What absolutely does not work is trying to just feed a set of documents into fine tuning. I personally have proven that dozens of times because I had a client who is determined to do it. He has been mislead.
What it will do is learn the patterns that are in those documents.
2. Ask HN: What are good books/blogs to read for a first time CTO?
Top comment by elmarschraml
Depends a lot on your situation:
Does "CTO" mean you are the tech lead of a small (single team) engineering organization? Then everything written for staff engineers applies. E.g I've heard good things about "Staff engineer's path" by Tanya Reilly.
Does "CTO" mean you are leading an org that is too large to be hands-on with tech, and need to build an effective structure and culture? Then I second the recommendation for "an elegant puzzle" by Will Larson.
Or does "CTO" mean that you switched from being an engineer to managing a team of engineers? Then everything for new managers applies, for starters I'd recommend "Becoming an effective software engineering manager" by James Stanier, or "Engineering management for the rest of us" by Sarah Drasner.
For some good general material, I'd also recommend the resources that Gergely Orosz makes available for subscribers to his "pragmatic engineer" newsletter. Those are templates for the kind of documents and processes you will most likely need - if you're new to the role, you will not go too wrong by using them, and if you want to create your own they are excellent starting points.
3. Ask HN: Programming Courses for Experienced Coders?
Top comment by onetimeuse92304
There is a general lack of advanced materials in software engineering. This is one area where market economy works against common good, unfortunately. People who have the knowledge and want to capitalise on it are highly incentivised to preparing beginner level materials and maybe a tiny bit to prepare the something for people who already have a bit of knowledge. After that, the number of people who are your potential target drops off sharply and you would have to increase the price dramatically. And it happens that people will not buy a $200 book even if it had $100k worth knowledge in it.
There are some positions available though. For example, I recently found Let over Lambda by Doug Hoyte. Excellent book if you are that kind of programmer.
Another problem with advanced materials is that frequently people are even unable to recognise them as advanced or valuable. It is easy to look at and evaluate things you have experience with, it is much more difficult to impossible to do the same with stuff that is beyond your experience (also called The Blub Paradox: https://www.paulgraham.com/avg.html).
Personally, I just read a lot. I try to keep an open mind and even if I don't agree fully with the contents of the book, I try to use the ideas as inspirations for my own thinking. You can read codebases written by other people and learn new ideas. Over time, you gather a library of solutions to problems and knowledge of when to and when not to apply them.
If you do enough of it, you will find useful knowledge in unusual places.
4. Ask HN: Share your favorite software blog posts of 2023
Top comment by ya3r
Bicycle by Bartosz Ciechanowski: https://ciechanow.ski/bicycle/ - HN post: https://news.ycombinator.com/item?id=35343495
Bartosz has many great blog posts, maybe most famously the one on mechanical watches published in 2022: https://ciechanow.ski/mechanical-watch/
HN posts for Bartosz's blog: https://news.ycombinator.com/from?site=ciechanow.ski
5. Ask HN: What are your predictions for 2024?
Top comment by caprock
I'm generally optimistic about the next five years in the United States.
Change has already started, but 2024 will make it more clear. Several societal pendulums are slowing, will reach apex, and reverse course.
* Optimism will regain footing
* Belief in positive
application of technologies which shape earth and society, will increase
* Conversation and information flows will recenter on individual responsibilities in service to community
This has already started in tech, business, and culture. It's partly a response to the deflation of the 2022 growth bubble and to political instability. See American Dynamism and e/acc for narrow domain examples.
The elections will act as a forcing function, and muddy the waters a bit. However, the political world will then be a lagging indicator. People will refactor their world models regardless of who wins.
We are a caterpillar. It's going to be intense as we build the cocoon and transform over the next year or few. But then...
6. Ask HN: Favorite Podcast Episodes of 2023?
Top comment by benwoodruff
SBF on Odd Lots describing yield farming [1]. For context, this was before the FTX meltdown or any allegations about Alameda using FTX customer funds. Was honestly stunning to hear someone unintentionally explaining their business as a ponzi scheme.
[1] - https://podcasts.apple.com/us/podcast/sam-bankman-fried-and-...
7. Ask HN: Good examples of fault-tolerant Erlang code?
Top comment by toast0
If you follow OTP design principles, you end up with a supervision tree, and a lot of code like...
ok = do_something_that_might_fail()
There's two issues you might rapidly encounter.
1) if a supervised process restarts too many times in an interval, the supervisor will stop (and presumably restart), and that cascades up to potentially your node stopping. This is by design, and has good reasons, but might not be expected and might not be a good fit for larger nodes running many things.
2) if your process crashes, its message queue (mailbox) is discarded, and if you were sending to a process registered by name or process group (pg), the name is now unregistered. This means a service process crashing will discard several requests; the one in progress which is probably fine (it crashed after all), but also others that could have been serviced. In my experience, you end up wanting to catch errors in service processes, log them, and move on to the next request, so you don't lose unrelated requests. Depending on your application, a restart might be better, or you might run each request in a fresh process for isolation... Lots of ways to manage this.
8. Ask HN: Should I medicate my ADHD?
Top comment by saaaaaam
While anecdotal advice can be useful to contextualise stuff like this I would strongly be to urge you not to base this decision on advice from semi-anonymous people, and get a proper psychiatric assessment of your ADHD if you are able to. If you were diagnosed as a child it’s entirely possible that the nature of your ADHD has evolved over time. It used to be thought people “grow out of it” but this seems less well supported clinically now.
However, the impact and nature of ADHD is very much down to the individual and it sounds like you have either developed strategies to harness your ADHD or you may have ADHD impulsive/hyperactive without significant attention deficit/distraction. There is not “one type” of ADHD and this means that two people diagnosed with ADHD can have very different experience of things.
Before starting medication you should get an adult ADHD assessment and in particular the Qb Test. For me the Qb Test was particularly useful in diagnosis.
Diagnosis of, and thinking around ADHD has come on significantly in the past couple of decades so it’s likely that reassessment as an adult could have upside for you - even if you don’t end up going down a medication route - by giving you greater insight into the specific nature of your ADHD.
FWIW I was undiagnosed as a child but the symptoms were very clear - very hyperactive, very inattentive in class but due to good levels of general intelligence, ability to recall texts in detail after reading, and ability to hyperfocus (writing essays last minute, cramming for exams the day before) got high marks throughout school and college - but at significant cost to my overall wellbeing.
I was then diagnosed as an adult and have significantly benefited from medication.
As someone who used to use hyperfocus as a “superpower” and used to be able to deliver huge pieces of work by using it I fooled myself into thinking there wasn’t a problem, even though I knew there was. I actually get more done with less need to hyperfocus and feel less stressed and less “backfooted” now that I take medication. I’ve not lost the ability to hyperfocus but am bale to combine it with the ability to manage and plan, something I did not used to be able to do.
9. Ask HN: Questions for two 40 year IBM Mainframe repair and programmers
Top comment by markus_zhang
I'd very much like the two gentlemen to sit down and write a memoir about their mainframe days. The older technologies are particularly fascinating. It doesn't have to be a book, but if it is then they can crowd fund.
10. Ask HN: What is a quote that permanently changed the way you think?
Top comment by Quinzel
The one that sticks with me is “show me your friends and I’ll show you your future”. (I’m sorry I can’t remember where I heard the quote).
I grew up in a place where a lot of people were addicted to drugs and alcohol and where poverty was pretty bad. I would see how they all encouraged each other to continue on their paths to self-destruction through their friendships, and I guess I was already quite picky about who my friends were but after hearing that quote I have definitely become more picky about friendships.
Then someone else once made a remark to me (which I can’t remember word for word to quote it) about watching crabs in a bucket, how they could all escape but they keep pulling each other down.
Those two quotes/concepts together have meant for me, friendship is never really a strong priority. I’m really not interested in feeling popular, and I don’t tend to struggle with loneliness because I see people all the time. I also have family members that I’m extremely close to. Especially my brother - who just gets me.
I grew up a bit of a loner and I struggled to make friends as a kid because I was super shy. People never realised I was shy because I was also super loud and talkative when I found people I felt comfortable talking to… but I generally won’t talk to new people unless they talk to me first. So maybe those quotes resonated because I was already destined to be a bit of a loner anyway and extremely selective with my friends… but I think when you think about how friendships can start to define the environments you choose to put yourself in, I think it’s a pretty life changing quote because perhaps you could change your life for the better, by changing the people you surround yourself by.