literatures

papers i co-authored

approaches to word embeddings

this project explores theoretical frameworks for understanding the empirical success of various word embedding methods, such as pmi, word2vec, and glove. the rand-walk model combines a log-linear topic model with a random walk over a latent discourse space, offering insights into the linear structures observed in word embeddings and their ability to solve analogies. the research highlights the importance of low dimensionality and isotropy in capturing semantic relationships between words.

music genre classification

this project focuses on improving music genre classification, crucial for digital platforms like spotify. using the gtzan dataset, we transformed 30-second audio clips into short-time fourier transform (stft) and mel-scale spectrograms. our approach utilized densely connected convolutional networks (densenet) and introduced adaptive training techniques to enhance classification accuracy and prevent overfitting. the model’s performance was comprehensively evaluated using auroc metrics.

favorite papers

• a simple network module for relational reasoning
• a tutorial introduction to the minimum description length principle
• attention is all you need
• chapter 14: kolmogorov complexity
• cs231n: convolutional neural networks for visual recognition
• deep residual learning for image recognition
• deep speech 2: end-to-end speech recognition in english and mandarin
• gpipe: efficient training of giant neural networks using pipeline parallelism
• identity mappings in deep residual networks
• imagenet classification with deep convolutional neural networks
• keep neural networks simple by minimizing description length of weights
• machine super intelligence
• multi-scale context aggregation by dilated convolutions
• neural machine translation: by jointly learning to align and translate
• neural message passing for quantum chemistry
• neural turing machines
• order matters: sequence to sequence for sets
• pointer networks
• quantifying the rise and fall of complexity in closed systems: the coffee automaton
• recurrent neural network regularization
• relational recurrent neural networks
• scaling laws for neural language models
• the annotated transformer
• the first law of complexodynamics
• the unreasonable effectiveness of recurrent neural networks
• understanding lstm networks
• variational lossy autoencoder

must-reads

• algorithms: grokking algorithms
• clean code
• deep learning with python
• designing data-intensive applications: the big ideas behind reliable, scalable, and maintainable systems
• discrete math and its applications
• grokking deep learning
• handbook of data compression (5th edition)
• operating systems: three easy pieces
• programming massively parallel processors: a hands-on approach
• the pragmatic programmer