Skip to content

Conversation

@bpkroth
Copy link
Contributor

@bpkroth bpkroth commented Jan 19, 2024

Work split out from #633

  • Reorgs the unit test fixtures for reuse by mlos_viz (future PR)
  • Removes TunablesGroups from Storage classes. Currently unnecessary, belongs in Experiment, and causes inconsistencies with ExperimentData interactions.
  • Adjusts the initialization code to match.

Work split out from microsoft#633

- Reorgs the unit test fixtures for reuse by mlos_viz (future PR)
- Removes TunablesGroups from Storage classes.
  Currently unnecessary, belongs in Experiment, and causes inconsistencies with ExperimentData interactions.
- Adjusts the initialization code to match.
@bpkroth bpkroth marked this pull request as draft January 22, 2024 19:30
@bpkroth bpkroth marked this pull request as ready for review January 22, 2024 20:45
@motus motus merged commit c786a25 into microsoft:main Jan 23, 2024
bpkroth added a commit that referenced this pull request Jan 23, 2024
…rty for `TrialData` and `ExperimentData` (#648)

Useful for grouping trials by the config they used. In use by upcoming
#633 for generating graphs with variance error bars for repeated
configs.

Also refactors a number of other things:
- Standardize on `experiment_id` instead of `exp_id`
  (but not the db schema for now)
- Standardize on `tunable_config_id` instead of `config_id` in the API
since we also call it `tunable_config` for the object fetching property
to distinguish from the `config` dict used internally.
  (but not the db schema for now)
- Rework the idea of `TunableConfigTrialGroup` as an object inaddition
to an ID (further methods can be added later to move back and forth
between types when doing interactive analysis).
- Rework the idea of a `TunableConfig` as an object for fetching tunable
value assignments (similar justification - easier grouping in the future
by fetching trial across experiments based on config - eventually could
be used to house the experiment merge logic).
- Rename `results` APIs to `results_df` (similar for others that return
`pandas.DataFrame`) to match the `results_dict` that return `dict`
- Refactor test fixtures to match other styles and for future use (moved
to #644).
- Expand tests

NOTE:
- We cut a new version with this commit since there are potentially
breaking API changes (e.g., `results` -> to `results_df` and `exp_id` ->
`experiment_id`).

Currently builds off of #644 and splits work out of #633

---------

Co-authored-by: Sergiy Matusevych <[email protected]>
@bpkroth bpkroth deleted the remove-tunables-from-storage-classes branch February 27, 2024 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants