Skip to content

Add the option to export the incidence matrix in sparse format#913

Closed
harrymvr wants to merge 1 commit intonetworkx:masterfrom
harrymvr:sparse-incidence-matrix
Closed

Add the option to export the incidence matrix in sparse format#913
harrymvr wants to merge 1 commit intonetworkx:masterfrom
harrymvr:sparse-incidence-matrix

Conversation

@harrymvr
Copy link
Copy Markdown
Contributor

The incidence matrix is huge in size (#nodes x #edges) but really sparse, since there are only two non-zeros elements per column. The way the code is right now does not allow to export the incidence matrix of a large graph (million nodes by million edges) as it initializes the result as a dense matrix full of zeros. However, using a sparse representation, as the one in scipy.sparse.lil_matrix, would make it work.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage decreased (-0%) when pulling a90db71 on harrymvr:sparse-incidence-matrix into 7d9682a on networkx:master.

@ghost ghost assigned hagberg Jul 24, 2013
@hagberg
Copy link
Copy Markdown
Member

hagberg commented Jul 28, 2013

This is a good idea.

In fact we could have sparse matrix versions of other functions too e.g. adjacency_matrix() and laplacian_matrix().

There are some design choices for the names and arguments for those functions and I don't think we have come to a decision on how to do it. We could do as you suggest and add a keyword argument whether you want a sparse or dense matrix format. Or we could use two different functions like we do in convert.py for generating adjacency matrices. Once you have a sparse matrix S you can easily get a dense matrix by calling D = S.todense() so that is another option (only have sparse matrix formats).

So before we incorporate this code I'd like to come to a consensus on how to proceed with sparse matrices. We can comment here or maybe the mailing list too.

@dschult
Copy link
Copy Markdown
Member

dschult commented Aug 1, 2013

I think we should switch to using all sparse format matrices. My reasons
are similar to the desire to use iterators wherever we can. It is easy to
switch to dense matrices if needed and for large networks a dense matrix
doesn't make sense. I think all our routines that require input of a
matrix work with either sparse or dense, but I'm not sure of that.

So +1 for sparse matrix. (Would changing them all to being sparse have
to wait for version 2.0? If so, maybe a short-term keyword argument would
be worthwhile.)

Dan

On Sun, Jul 28, 2013 at 10:13 AM, Aric Hagberg notifications@github.comwrote:

This is a good idea.

In fact we could have sparse matrix versions of other functions too e.g.
adjacency_matrix() and laplacian_matrix().

There are some design choices for the names and arguments for those
functions and I don't think we have come to a decision on how to do it. We
could do as you suggest and add a keyword argument whether you want a
sparse or dense matrix format. Or we could use two different functions like
we do in convert.py for generating adjacency matrices. Once you have a
sparse matrix S you can easily get a dense matrix by calling D =
S.todense() so that is another option (only have sparse matrix formats).

So before we incorporate this code I'd like to come to a consensus on how
to proceed with sparse matrices. We can comment here or maybe the mailing
list too.


Reply to this email directly or view it on GitHubhttps://github.com//pull/913#issuecomment-21684094
.

@hagberg
Copy link
Copy Markdown
Member

hagberg commented Aug 24, 2013

I'm +1 on only having sparse matrix formats and letting users use todense() to get a dense numpy matrix.

It might be worth looking at how that impacts performance of some algorithms. For small graphs the sparse matrices might be slower to generate and convert.

@hagberg hagberg mentioned this pull request Nov 27, 2013
@hagberg hagberg closed this Dec 15, 2013
@harrymvr harrymvr deleted the sparse-incidence-matrix branch April 7, 2014 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants