likely llc

I penned this up in the last month. I had to stop this project, I was living in Mexico on and off for 2-3 years in Airbnb’s and I was offered a job by SAS which provided me enough income to return to USA, and so I did.

This method could be used in the case where we have two design matrices, and we’d like to see association of groups of both design matrices covariates (groups) with an outcome of intereste (whatever, likelihood of purchasing an elephant). For some reason group lasso wasn’t working, either issues with software or some issue with the structure of the dataset, I don’t remember. Issues with open source software packages in R happen a lot, either they weren’t developed to be production code, or it was done by a grad student who had only considered their case, and not edge cases, underflow or practical constraints (such as model selection which I’ll hint on later). This is why we have production open source software (linux is a good example, as well as the popular Bayesian inference or ML packages we love (Stan, PyMC, pytorch, tensorflow), and then the proprietary ones have great devs (SAS).

What the researcher ended up using was a Canonical Correlation Analysis (CCA) like method, who’s algorithm was a convex optimization algorithm. A few issues I didn’t like, is that, like k-means clustering, the solution space was multimodal, so the result would be different depending on the parameters you initialize it with. And, moreover, the variable selection method. It suggested an n choose k type variable selection method, optimizing some information criteria, but the cost of choosing (1000 choose 3), is clearly computationally prohibitive. You’d think someone famous at a large university wouldn’t overlook this… but anyway.

So data we have looks like this and , and we’re investigating on a regression problem, we use the standardized cross covariance matrix, and use a simple clustering algorithm like k-means, and then after we obtain cluster labels with clusters, we can then use these labels as a covariate in a regression model, where we can use a hierarchical model (Bayesian) which would be simple to implement in Stan, or a multilevel model. So it’s kinda like, “Spectral Clustering with Some Extra Steps (SCSES)”. A bonus of SCSES is that we could adjust for additional covariates. For example, if there’s higher concentration or presence (what’s the metric?) of certain metabolites or proteins given a person’s diet, we can adjust for that, meaning eating a bunch of sugar could confound with the amount of a certain amount of proteins or metabolytes or cluster k’s association with diabetus, or something.

Any way, I’m waving my hand at some details, but there should be more in my notes, below. Feel free to use and see if you can find any way to validate this method (cross validation, out of sample, ability to predict outcome of interest given these clusters). Credit is appreciated but not necessary (or do I even want credit?). I would be interested to see what’s up.

Per usual, I’m available to apply this to any dataset or come up with new solutions if anyone’s interested. Drop me a line at the home page, it goes to one of my emails.

Here are my notes, done from memory without internet or a book, should give you the idea.

Again, really not worth writing up unless it will be published, or I’m paid, or there’s a dataset to use it on, which are all prohibitive parameters as of now, so here you go: