The Wayback Machine - https://web.archive.org/web/20201209110609/https://github.com/tensorflow/java/pull/166
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indexing API #166

Open
wants to merge 8 commits into
base: master
from
Open

Indexing API #166

wants to merge 8 commits into from

Conversation

@rnett
Copy link
Contributor

@rnett rnett commented Dec 7, 2020

Fixes #164.

One question: Should I also make endpoints named get and set? It's more discoverable to people coming from Python (who usually use [] rather than the stridedSlice op).

There's examples in the Index javadoc.

rnett added 7 commits Dec 7, 2020
Signed-off-by: Ryan Nett <[email protected]>
Signed-off-by: Ryan Nett <[email protected]>
codegen
Signed-off-by: Ryan Nett <[email protected]>
op test
Signed-off-by: Ryan Nett <[email protected]>
fix test
Signed-off-by: Ryan Nett <[email protected]>
Signed-off-by: Ryan Nett <[email protected]>
Signed-off-by: Ryan Nett <[email protected]>
Copy link
Collaborator

@Craigacp Craigacp left a comment

More generally these indices don't seem to link up with the indexing operations already available in ndarray, which might prove more confusing as the ndarrays move closer to Tensors given Karl's current refactorings.

* <p>
* - An ellipsis (...) using {@link Index#ellipses()}. Ellipses are used to imply zero or more
* dimensions of full-dimension selection and are produced using
* `ellipsis_mask`. For example, `foo[...]` is the identity slice.

This comment has been minimized.

@Craigacp

Craigacp Dec 9, 2020
Collaborator

We don't have this kind of overloading in Java, so does it make sense to describe it here in the Java Ops?

This comment has been minimized.

@rnett

rnett Dec 9, 2020
Author Contributor

I copied most of this from the stridedSlice docs and tried to keep it as close as possible. I talk enough about the Python and Java equivalents that I think it's understood to be stridedSlice(foo, Index.ellipsis()). I'm kind of using it as a generic "slice notation" rather than specifically referencing the Python syntax.

Signed-off-by: Ryan Nett <[email protected]>
@rnett
Copy link
Contributor Author

@rnett rnett commented Dec 9, 2020

I looked at that, there's a few concepts that are impossible to map into stridedSlice, which as far as I know is the only Op we could use for this. In particular, seq and hyperslab. The ndarray API also has no ellipses or newaxis.

If we do want to unify APIs, I think it would be easier to map this API onto the current ndarray one the visa versa. To map the ndarray indices onto Operands we'd either have to remove index options, throw runtime exceptions for unsupported ones, or have a TensorIndex or similar sub-interface. If I was doing it I'd probably have a shared Index interface with TensorIndex and NDArrayIndex for the unsupported ones. Although it seems like you could add operations to the ndarray indexing.

@Craigacp
Copy link
Collaborator

@Craigacp Craigacp commented Dec 9, 2020

I think extending the ndarray API to cover the gaps would be worthwhile, and then we can throw exceptions for things which fail to map from ndarray into TF ops.

@rnett
Copy link
Contributor Author

@rnett rnett commented Dec 9, 2020

There's some odd things in the ndarray API like At not supporting negative indices, Range not having a stride or supporting negative indices, and having flip, even, and odd separate from [::-1], [::2], and [1::2] (although I like having functions for them, I don't think they need to be their own classes). Do you think I should bring the ndarray indexing a bit closer to Numpy? It would make the differences between the ndarray indexing and the tensor indexing less confusing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants
You can’t perform that action at this time.