On Sat, 26 Oct 2019, Artur Tarassow wrote:
Am 26.10.19 um 17:57 schrieb Riccardo (Jack) Lucchetti:
> On Sat, 26 Oct 2019, Sven Schreiber wrote:
>> In particular -it's not a secret- Artur and I have been working on a
>> partial wrapper for R's glmnet package. It's not finished yet, but
>> probably would be ready this Winter.
> That would be excellent.
> However, I consider it of the highest importance to have a native
> implementation of LASSO (at least, but elastic net would be nice too) ready
> as soon as time allows. Allin and I talked about it briefly in June. I
> guess we could start thinking about it.
Having a built-in LASSO would be cool of course. Some time ago I was looking
for existent C-based implementations. On github one find some existent
libraries. Of course: No idea how well and accurate they work and whether
stuff is properly tested. But maybe there exists some well-written library
similar to libsvm.
Still, the glmnet library offers support for several types of models:
gaussian, multinomial, cox etc. Not sure whether you plan to support all of
these - I guess not as this involves a lot of work. Hence, the the glmnet
wrapper could still support _additional _ model types.
It so happens that over the last few days I've been reviewing where
Jack and I got to with lasso this past summer, with a possible
function package in mind.
I'm currently testing against glmnet, trying to figure out where
differences are coming from and whether there's anything wrong with
the method we're using (the Alternating Direction Method of
Multipliers, from Stephen Boyd et al.) I think what we have is
sound, but the confusing thing is comparing lambdas: the Boyd et al
algorithm takes as argument the fraction, s, of the lambda value
lmax that drives all coefficients to zero while glmnet (seems to)
take the "raw" lambda value. I'm not totally clear on how lmax
should be calculated and/or exactly what glmnet is doing.
Anyway, seems to me a reliable lasso implementation in hansl for
gaussian linear models would be a useful first step. Right now I'm
not too bothered about trying to emulate elnet's lasso/ridge hybrid,
or the non-gaussian variants supported by glmnet.