Weekly AI/ML & Biotech Digest — Aug 17 to Aug 23, 2026

It have been quite a busy time since a recent family visiting trip. I only get a chance to read a few articles in the past week. Here is one that seems interesting.

  • ★ Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length — The quadratic complexity and weak length extrapolation of Transformers limits their ability to scale to long sequences, and while sub-quadratic solutions like linear attention and state space models exist, they empirically underperform Transformers in pretraining efficiency and downstream task accuracy.
  • Another interesting article on synthetic biology, where researchers created a synthetic cell that puts user closer to building life from scratch. Here is a comment article from Science.

Comments

Post a Comment

Popular posts from this blog

Controlled Sampling in High-Dimensional Latent Spaces for Protein Design

A fundamental challenge in generative artificial intelligence involves sampling from carefully constructed high-dimensional latent spaces and utilizing these samples as inputs to decoder networks for generating novel entities. In the context of computational protein design, this process typically involves sampling regions within protein embedding spaces where specific biochemical properties are anticipated, such as enhanced binding affinity or improved developability characteristics in therapeutic antibodies. The sample-decode paradigm presents several significant technical challenges that must be addressed for effective protein generation. First, determining the optimal sampling distance from training datasets remains a critical consideration—sampling too close may limit diversity, while sampling too far may compromise biological relevance. Second, identifying which directions in the latent space merit more extensive exploration requires careful consideration of the underlying protein...
Read more

First Impressions: Claude Code as an AI Coding Assistant

I was really impressed by Claude Code. For just $5 and under an hour, Claude scanned my project repository and generated the skeleton of a new functionality based on my prompt. While the code wasn’t error-free, the overall logic was sound. Reaching that point on my own would have taken me hours, if not days. About half of the errors could be resolved with additional prompting, and I spent roughly another hour fixing the remaining bugs and testing. In total, I had working, productive code in about four hours—work that would normally take me closer to twenty hours. In other words, that $5 effectively saved me 15 hours of effort!
Read more

Google Colab as a Cloud GPU Platform: Notes and First Impressions

After tracking Nvidia’s RTX 50-series cards for a while, I still feel they are too expensive and likely not powerful enough to fully support my AI enthusiasm. As a result, I’ve decided to explore alternatives, particularly cloud solutions. Google Colab has turned out to be a very appealing option for now. It provides a Jupyter Notebook interface backed by CPUs, GPUs, and TPUs managed by Google. There is a free tier that offers limited GPU access (availability-based). During my test run this Sunday morning, I was able to access a GPU with 16 GB of memory. Paid users receive access to more powerful GPU and CPU resources. This makes Colab a viable alternative to running an on-premise AI lab—provided you’re comfortable with Google hosting your data and models. I haven’t checked AWS’s GPU offerings yet, but based on my prior EC2 experience, I feel Colab is much more convenient. Colab simplifies many aspects (from billing to system management), allowing you, as an AI scientist, to focus on m...
Read more