Out of memory, or gradient accumulation for larger modelsWhen the model grows large and does not fit on a single device, and there are no more devices to spare, the common mitigation strategy is…Jan 31Jan 31
Relative positional embedding for any attention mechanismIn Shaw et al. (2018), the authors introduce relative positional embedding for self-attention in transformer models, and in Huang et al…Jan 171Jan 171
Breaking sticks, or estimation of probability distributions using the Dirichlet processRecall the last time you wanted to understand the distribution of given data. One alternative was to plot a histogram. However, it resulted…Jan 25, 2021Jan 25, 2021
Heteroscedastic Gaussian process regressionGaussian process regression is a nonparametric Bayesian technique for modeling relationships between variables of interest. The vast…Jun 22, 2020Jun 22, 2020
What is the easiest way to compare two data sets?One has probably come across this problem numerous times. There are two versions of a tabular data set with a lot of columns of different…Apr 10, 2020Apr 10, 2020
Bayesian inference of the net promoter score via multilevel regression with poststratificationCustomer surveys are naturally prone to biases. One prominent example is participation bias, which arises when individuals decide not to…Feb 3, 2020Feb 3, 2020
Ingestion of sequential data from BigQuery into TensorFlowHow hard can it be to ingest sequential data into a TensorFlow model? As always, the answer is, “It depends.” Where are the sequences in…Nov 8, 2019Nov 8, 2019
Sample size determination using historical data and simulationIn order to test a hypothesis, one has to design and execute an adequate experiment. Typically, it is neither feasible nor desirable to…Sep 25, 2019Sep 25, 2019
A Bayesian approach to the inference of the net promoter scoreThe net promoter score is a widely adopted metric for gauging customers’ satisfaction with a product. The popularity of the score is…Aug 19, 2019Aug 19, 2019
Interactive notebooks in tightly sealed disposable containersIt is truly amazing how interactive notebooks-where a narrative in a spoken language is entwined with executable chunks of code in a…Jul 24, 2019Jul 24, 2019
