Add LDLFactorizations.jl as a package extension

[odow]

Closes #1971
Closes #2931

[odow]

I don't understand how the default fallback is covered. I need to investigate more. It must be getting hit by a test before we load LDLFactorizations, but that seems flakey. I need to add an explicit test.

[odow]

Another thing to consider is that this does add a dependency to MOI with respect to compat bounds. But I think we're okay to take it on. cc @amontoison

[mlubin]

Maybe we should ask for a 1.0 tag on LDLFactorizations?

[amontoison]

Did you try the ldlt from CHOLMOD?
We don't plan to break LDLFactorizations.jl anytime soon.
If I do a release 1.0, I need to update a bunch of Project.toml in JSO, is it a big deal to keep the compat entry 0.10?

[odow]

It doesn't support zero pivots:

julia> import SparseArrays

julia> A = [1.0 1.0; 1.0 1.0]
2×2 Matrix{Float64}:
 1.0  1.0
 1.0  1.0

julia> Q = SparseArrays.sparse(A);

julia> LinearAlgebra.ldlt(Q)
ERROR: ZeroPivotException: factorization encountered one or more zero pivots. Consider switching to a pivoted LU factorization.
Stacktrace:

[amontoison]

It depends what you want to achieve at the end but why not calling again cholesky / ldlt with the keyword argument shift set to a value near epsilon machine if you have this error?

[odow]

Given $x^\top Q x$ we want $Q = A^\top A$ so we can do $|| Ax ||_2^2$. We'd prefer not to have small numerical values because these show up in the cone constraint.