Skip to content

Commit 1771f66

Browse files
dmartinoltmihalacredhatHameedlokeshranginenifranciscojavierarceo
authored
feat: Feast Security Model (aka RBAC) (#4380)
* initial commit Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fixed linting issues (but 1) Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * deleted AuthzedResource and moved types to the Permission class Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * using pytest.mark.parametrize tests Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * moved decorator to decorator module Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * parametrized decision tests Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Added matcher and action modules. Added global assert_permissions function Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fixed linting error Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Managing with_subclasses flag and overriding it in case it's an abstract class like DataSource Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Permission includes a single Policy Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * completed docstrings for permissions package Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fixed inter issues Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Changed roles matching rule from "all" to "any" Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Introducing permission framework and authorization manager in user guide (to be continued after the code is consolidated) Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * removed test code Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * hiding sensitive data (false positive, anyway) Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Added filter_only flag to assert_permissions and returning a list of filtered resources instead of PermissionError Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * added the option to return the single resource, or None Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * separate validating functions: assert_permission and filtered_resources Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Store and Manage permissions in the Registry Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Applied review comments Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Store and Manage permissions in the Registry - Fixes to code - Made test case broader Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Store and Manage permissions in the Registry - Fixed incorrectly recognized linter error Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Store and Manage permissions in the Registry - Added test - Fixed missing property to permission - Changed code following review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Store and Manage permissions in the Registry Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Store and Manage permissions in the Registry - Fixes to code - Made test case broader Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Store and Manage permissions in the Registry - Fixed incorrectly recognized linter error Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Store and Manage permissions in the Registry - Added test - Fixed delete and apply permission Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * replaced aggregated actions with aliases for QUERY and WRITE and ALL Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Updated user guide Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Updated enum in proto Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Store and Manage permissions in the Registry - Fixed test errors following refactor - Added test Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Store and Manage permissions in the Registry - Removed redundant property - Added tags filter option to list_permissions Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Added permission assert check for registry server, offline server, online server functions Signed-off-by: Abdul Hameed <[email protected]> * Fix linter after rebase Signed-off-by: Abdul Hameed <[email protected]> * CLI command "feast permissions list" Added cli command permissions Added tags parameter to list_validation_references and list_saved_datasets in registry Added list_validation_references and list_saved_datasets apis to feature_store Added missing tags parameters to registry_server methods Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * CLI command "feast permissions list" - Changes following review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * CLI command "feast permissions list" - Changes following review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * added the documents reference for permissions for online, offline, registry server endpoints. Signed-off-by: Abdul Hameed <[email protected]> * Incorporating code review comments to parse the auth block from the f… (#36) * Incorporating code review comments to parse the auth block from the feature_store.yaml file. Signed-off-by: Lokesh Rangineni <[email protected]> * Incorporating code review comments - renaming type from k8 to kubernetes. Signed-off-by: Lokesh Rangineni <[email protected]> --------- Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * definition and integration of auth manager in feast offline and online servers Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * typo Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * duplicated if Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * renamed functions with long name Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * using User class instead of RoleManager (completely removed) Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Feed SecurityManager with Registry instance to fetch the actual permissions Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fixed linter Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * review comments Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fixed broken IT Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Adding registry server (UT to be completed) Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fix linter Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * passing auth manager type from config Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * used auth config to set auth manager type Signed-off-by: Abdul Hameed <[email protected]> * inject the user details Signed-off-by: Abdul Hameed <[email protected]> * created decorator function and applied to arrow function for injecting the user detail:wq Signed-off-by: Abdul Hameed <[email protected]> * code review fixes including the unit test and integration test as suggested Signed-off-by: Abdul Hameed <[email protected]> * Implementation of oidc client authentication. (#40) * Adding initial draft code to manage the oidc client authentication. Signed-off-by: Lokesh Rangineni <[email protected]> * Adding initial draft code to manage the oidc client authentication. Signed-off-by: Lokesh Rangineni <[email protected]> * Incorporating code review comments. Signed-off-by: Lokesh Rangineni <[email protected]> --------- Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Client module-grpc - Added authentication header for client grpc calls Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Client module-grpc - Made changes following code review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Client module-grpc - Made changes following code review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Client module-grpc - Made changes following code review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Client module-grpc - Made changes following code review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Client module-grpc - Made changes following code review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Client module-grpc - Made changes following code review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Client module-grpc - Made changes following code review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * added auth configuration for arrow flight client Signed-off-by: Abdul Hameed <[email protected]> * Client module-grpc - Made changes following code review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fix linter Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Propagating auth config to token parser in server init Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * adding headers and client_secret to token request Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * working E2E test of authenticated registy server Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * renamed test Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fixed broken test Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fix rebase issues Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fix rebase issues Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Adding the auth client documentations and unit testing for auth client code. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Adding the auth client documentations and unit testing for auth client code. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Incorporating code review comments. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Incorporating code review comments. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Introducing permission framework and authorization manager in user guide (to be continued after the code is consolidated) Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * CLI command "feast permissions list" - Added missing dependency Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Client module-grpc - Added missing auth header for calls to remote registry Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Fix auth tests with permissions - Made changes to enforcer ana security manager permission checking logic Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Fix auth tests with permissions - Made changes following review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Fix auth tests with permissions - Made changes following review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Fix auth tests with permissions - Made changes following review Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Moved the common fixtures to the root conftest.py or auth_permissions_util.py (#54) * Moved the common fixtures to the root conftest.py or auth_permissions_util.py Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> * Adding missed dependency and regenerated the requirements files. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> * Addinig missing changes from the original PR. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> --------- Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * added check and list-roles subcommands Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * typo Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * added comment in cli_utils to remind the original function from which this logic was derived Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * 1) Updating the existing integration test with auth permissions configurations. 2) Refactored the common code and moved to the util class and common conftest.py file. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Moved the common fixtures to the root conftest.py or auth_permissions_util.py Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Adding missed dependency and regenerated the requirements files. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * 1) Updating the existing integration test with auth permissions configurations. 2) Refactored the common code and moved to the util class and common conftest.py file. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * 1) Fixing an issue with the way getting markers after changing the fixture scope to module. Now looking up the markers coming from the entire module run. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Fixed bug in GetPermission API Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Permission CRUD test Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Added feast-rbac example Signed-off-by: Abdul Hameed <[email protected]> * Added support to read the token from enviroment variable to run from local Signed-off-by: Abdul Hameed <[email protected]> * Fix the header for arrow fligth Signed-off-by: Abdul Hameed <[email protected]> * fix the header issue Signed-off-by: Abdul Hameed <[email protected]> * added permissions apply file Signed-off-by: Abdul Hameed <[email protected]> * set the user in the grpc server Signed-off-by: Abdul Hameed <[email protected]> * added roles and updated permission with all roles Signed-off-by: Abdul Hameed <[email protected]> * updated chart to include the service account Signed-off-by: Abdul Hameed <[email protected]> * created client example with roles and updated installation/cleanup script Signed-off-by: Abdul Hameed <[email protected]> * rebased with master Signed-off-by: Abdul Hameed <[email protected]> * Moved the common fixtures to the root conftest.py or auth_permissions_util.py (#54) * Moved the common fixtures to the root conftest.py or auth_permissions_util.py Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> * Adding missed dependency and regenerated the requirements files. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> * Addinig missing changes from the original PR. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> --------- Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Fixed DecisionStrategy not persisted Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Fixed DecisionStrategy not persisted Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Fixed DecisionStrategy not persisted - Implemented review comments Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Revert "Fix decision strategy not saved" Signed-off-by: Abdul Hameed <[email protected]> * Dropped global decision strategy Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * updated rbac demo example Signed-off-by: Abdul Hameed <[email protected]> * Adding permissions directly instead of from the common place for the online read integration tests. Cleaned up some minor changes to fix the unpredictable issue with the feature server process. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Initial Draft version to the tests with remote offline server with OIDC authentication permissions. Happy path only. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Abstracting the specific code for Offline Permissions by creating new class for PermissionsEnvironment. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Formatting the python files using make format-python. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Separated the permissions for online, offline and registry servers. moved the fixtures scope accordingly as we can't reuse the permissions for all the test cases. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Separated the permissions for online, offline and registry servers. moved the fixtures scope accordingly as we can't reuse the permissions for all the test cases. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Created the grpc client auth header interceptor and removed the manual injection of the header. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Created the grpc client auth header interceptor and removed the manual injection of the header. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fix: java to proto failing - changed java_outer_classname for Permission.proto and Policy.proto - removed experimental optional from permission proto Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * CLI command "feast permissions list" Added cli command permissions Added tags parameter to list_validation_references and list_saved_datasets in registry Added list_validation_references and list_saved_datasets apis to feature_store Added missing tags parameters to registry_server methods Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Moved the common fixtures to the root conftest.py or auth_permissions_util.py (#54) * Moved the common fixtures to the root conftest.py or auth_permissions_util.py Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> * Adding missed dependency and regenerated the requirements files. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> * Addinig missing changes from the original PR. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> --------- Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fix: java to proto failing - changed java_outer_classname for Permission.proto and Policy.proto - removed experimental optional from permission proto Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Adding the extra writer permission to fix the integration test issue with offline server. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Try to fix java integration test - ModuleNotFoundError: No module named 'feast.permissions.server' Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fix java integration test - ModuleNotFoundError: No module named 'jwt' Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * fix java integration test - ModuleNotFoundError: No module named 'kubernetes' Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Adding missing permissions for offline store test cases - classes FileSource, FeatureService classes. (#64) Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Updating the offline integration test permissions. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * updated test.py file for rbac-example Signed-off-by: Abdul Hameed <[email protected]> * fix the DeleteFeatureView function to handle stream feature view type Signed-off-by: Abdul Hameed <[email protected]> * Updating permissions of the integration test cases to address code review comments and also check if the online_read integration test fixes. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Incorporating the code review comments from Francisco on upstream PR. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Update docs/getting-started/concepts/permission.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Update docs/getting-started/concepts/permission.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Update docs/getting-started/concepts/permission.md Co-authored-by: Francisco Arceo <[email protected]> Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Small fixes (#71) * Improved permission denial log Signed-off-by: Daniele Martinoli <[email protected]> * Added leeway option to accept tokens released in the past (up to 10") Signed-off-by: Daniele Martinoli <[email protected]> --------- Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * commented/removed oidc tests to verify integration test commented/removed test_auth_permission.py file Signed-off-by: Abdul Hameed <[email protected]> * Enabling the keycloak related integration tests and also initializing the keycloak only once in the entire run. Reduced the number of works and increased the duration as well. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Making number of workers back to 8 and enabled the test_remote_online_store_read Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Making number of workers to 4. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Incorporating the code review comments from Tornike to use @pytest.mark.xdist_group(name="keycloak"). Reverting number of markers from 4 to 8 for the make file target test-python-integration-local. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Reverting number of workers from 8 to 4. Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Reverting number of workers from 8 to 4. Reverting the marker @pytest.mark.xdist_group(name="keycloak") Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Reverting number of workers from 8 to 4 for make target test-python-integration-local Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Added the arrow flight interceptor to inject the auth header. (#68) * * Added the arrow flight interceptor to inject the auth header. * Injecting grpc interceptor if it is needed when auth type is not NO_AUTH. Signed-off-by: Lokesh Rangineni <[email protected]> * Fixing the failing integration test cases by setting the header in binary format. Signed-off-by: Lokesh Rangineni <[email protected]> * Refactored method and moved to factory class to incorporate code review comment. Fixed lint error by removing the type of port. and other minor changes. Signed-off-by: Lokesh Rangineni <[email protected]> * Incorproating code review comments from Daniel. Signed-off-by: Lokesh Rangineni <[email protected]> --------- Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * removed with_subclasses option (it's the default and unique behavior) Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * a full, minimal, reproducible example of the RBAC feature Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Add missing required_tags to permission object and cli info - Add missing required_tags to permission object - added required_tags to cli info Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Fixed the registry apply function assertation Signed-off-by: Abdul Hameed <[email protected]> * removed the examples Signed-off-by: Abdul Hameed <[email protected]> * Integrated comment Signed-off-by: Daniele Martinoli <[email protected]> * removed the firebase depdency and fix the doc conflicts Signed-off-by: Abdul Hameed <[email protected]> * Introducing permission framework and authorization manager in user guide (to be continued after the code is consolidated) Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> * Permission resources miss the created_timestamp and last_updated_timestamp fields Signed-off-by: Theodor Mihalache <[email protected]> * remove error incase if user has no roles assinged incase unthorized user Signed-off-by: Abdul Hameed <[email protected]> * renamed READ action to DESCRIBE Signed-off-by: Daniele Martinoli <[email protected]> * Specified authorization manager and authorization configuration Signed-off-by: Daniele Martinoli <[email protected]> * fix the linter and remove subclass from doc Signed-off-by: Abdul Hameed <[email protected]> * addressed the pr reivew comments Signed-off-by: Abdul Hameed <[email protected]> * Incorporating code review comment and this file is not needed. Signed-off-by: Lokesh Rangineni <[email protected]> * Addressed the review comments on the PR Signed-off-by: Abdul Hameed <[email protected]> * Reducing the markers from 8 to 4 to see if it fixes the issues with memory. Signed-off-by: Lokesh Rangineni <[email protected]> * addresses feedback on rbac doc Signed-off-by: Abdul Hameed <[email protected]> * rename action name from QUERY to READ Signed-off-by: Abdul Hameed <[email protected]> * fix the doc to replace query with read Signed-off-by: Abdul Hameed <[email protected]> --------- Signed-off-by: Daniele Martinoli <[email protected]> Signed-off-by: Abdul Hameed <[email protected]> Signed-off-by: Theodor Mihalache <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Signed-off-by: Lokesh Rangineni <[email protected]> Co-authored-by: Theodor Mihalache <[email protected]> Co-authored-by: Abdul Hameed <[email protected]> Co-authored-by: lokeshrangineni <[email protected]> Co-authored-by: Lokesh Rangineni <[email protected]> Co-authored-by: Francisco Arceo <[email protected]>
1 parent 46655f0 commit 1771f66

File tree

116 files changed

+6905
-329
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

116 files changed

+6905
-329
lines changed

Makefile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -86,14 +86,14 @@ test-python-unit:
8686
python -m pytest -n 8 --color=yes sdk/python/tests
8787

8888
test-python-integration:
89-
python -m pytest -n 8 --integration --color=yes --durations=10 --timeout=1200 --timeout_method=thread \
89+
python -m pytest -n 4 --integration --color=yes --durations=10 --timeout=1200 --timeout_method=thread \
9090
-k "(not snowflake or not test_historical_features_main)" \
9191
sdk/python/tests
9292

9393
test-python-integration-local:
9494
FEAST_IS_LOCAL_TEST=True \
9595
FEAST_LOCAL_ONLINE_CONTAINER=True \
96-
python -m pytest -n 8 --color=yes --integration --durations=5 --dist loadgroup \
96+
python -m pytest -n 4 --color=yes --integration --durations=10 --timeout=1200 --timeout_method=thread --dist loadgroup \
9797
-k "not test_lambda_materialization and not test_snowflake_materialization" \
9898
sdk/python/tests
9999

docs/SUMMARY.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
* [Feature retrieval](getting-started/concepts/feature-retrieval.md)
2424
* [Point-in-time joins](getting-started/concepts/point-in-time-joins.md)
2525
* [Registry](getting-started/concepts/registry.md)
26+
* [Role-Based Access Control (RBAC)](getting-started/architecture/rbac.md)
2627
* [\[Alpha\] Saved dataset](getting-started/concepts/dataset.md)
2728
* [Components](getting-started/components/README.md)
2829
* [Overview](getting-started/components/overview.md)

docs/getting-started/architecture/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,3 +23,7 @@
2323
{% content-ref url="model-inference.md" %}
2424
[model-inference.md](model-inference.md)
2525
{% endcontent-ref %}
26+
27+
{% content-ref url="rbac.md" %}
28+
[rbac.md](rbac.md)
29+
{% endcontent-ref %}

docs/getting-started/architecture/overview.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,7 @@ typically your Offline Store). We are exploring adding a default streaming engin
1717
write patterns](write-patterns.md) to your application
1818

1919
* We recommend [using Python](language.md) for your Feature Store microservice. As mentioned in the document, precomputing features is the recommended optimal path to ensure low latency performance. Reducing feature serving to a lightweight database lookup is the ideal pattern, which means the marginal overhead of Python should be tolerable. Because of this we believe the pros of Python outweigh the costs, as reimplementing feature logic is undesirable. Java and Go Clients are also available for online feature retrieval.
20+
21+
* [Role-Based Access Control (RBAC)](rbac.md) is a security mechanism that restricts access to resources based on the roles of individual users within an organization. In the context of the Feast, RBAC ensures that only authorized users or groups can access or modify specific resources, thereby maintaining data security and operational integrity.
22+
23+
246 KB
Loading
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Role-Based Access Control (RBAC) in Feast
2+
3+
## Introduction
4+
5+
Role-Based Access Control (RBAC) is a security mechanism that restricts access to resources based on the roles of individual users within an organization. In the context of the Feast, RBAC ensures that only authorized users or groups can access or modify specific resources, thereby maintaining data security and operational integrity.
6+
7+
## Functional Requirements
8+
9+
The RBAC implementation in Feast is designed to:
10+
11+
- **Assign Permissions**: Allow administrators to assign permissions for various operations and resources to users or groups based on their roles.
12+
- **Seamless Integration**: Integrate smoothly with existing business code without requiring significant modifications.
13+
- **Backward Compatibility**: Maintain support for non-authorized models as the default to ensure backward compatibility.
14+
15+
## Business Goals
16+
17+
The primary business goals of implementing RBAC in the Feast are:
18+
19+
1. **Feature Sharing**: Enable multiple teams to share the feature store while ensuring controlled access. This allows for collaborative work without compromising data security.
20+
2. **Access Control Management**: Prevent unauthorized access to team-specific resources and spaces, governing the operations that each user or group can perform.
21+
22+
## Reference Architecture
23+
24+
Feast operates as a collection of connected services, each enforcing authorization permissions. The architecture is designed as a distributed microservices system with the following key components:
25+
26+
- **Service Endpoints**: These enforce authorization permissions, ensuring that only authorized requests are processed.
27+
- **Client Integration**: Clients authenticate with feature servers by attaching authorization token to each request.
28+
- **Service-to-Service Communication**: This is always granted.
29+
30+
![rbac.jpg](rbac.jpg)
31+
32+
## Permission Model
33+
34+
The RBAC system in Feast uses a permission model that defines the following concepts:
35+
36+
- **Resource**: An object within Feast that needs to be secured against unauthorized access.
37+
- **Action**: A logical operation performed on a resource, such as Create, Describe, Update, Delete, Read, or write operations.
38+
- **Policy**: A set of rules that enforce authorization decisions on resources. The default implementation uses role-based policies.
39+
40+
41+
42+
## Authorization Architecture
43+
44+
The authorization architecture in Feast is built with the following components:
45+
46+
- **Token Extractor**: Extracts the authorization token from the request header.
47+
- **Token Parser**: Parses the token to retrieve user details.
48+
- **Policy Enforcer**: Validates the secured endpoint against the retrieved user details.
49+
- **Token Injector**: Adds the authorization token to each secured request header.
50+
51+
52+
53+
54+
55+
56+

docs/getting-started/components/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,7 @@
1919
{% content-ref url="provider.md" %}
2020
[provider.md](provider.md)
2121
{% endcontent-ref %}
22+
23+
{% content-ref url="authz_manager.md" %}
24+
[authz_manager.md](authz_manager.md)
25+
{% endcontent-ref %}
Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# Authorization Manager
2+
An Authorization Manager is an instance of the `AuthManager` class that is plugged into one of the Feast servers to extract user details from the current request and inject them into the [permissions](../../getting-started/concepts/permissions.md) framework.
3+
4+
{% hint style="info" %}
5+
**Note**: Feast does not provide authentication capabilities; it is the client's responsibility to manage the authentication token and pass it to
6+
the Feast server, which then validates the token and extracts user details from the configured authentication server.
7+
{% endhint %}
8+
9+
Two authorization managers are supported out-of-the-box:
10+
* One using a configurable OIDC server to extract the user details.
11+
* One using the Kubernetes RBAC resources to extract the user details.
12+
13+
These instances are created when the Feast servers are initialized, according to the authorization configuration defined in
14+
their own `feature_store.yaml`.
15+
16+
Feast servers and clients must have consistent authorization configuration, so that the client proxies can automatically inject
17+
the authorization tokens that the server can properly identify and use to enforce permission validations.
18+
19+
20+
## Design notes
21+
The server-side implementation of the authorization functionality is defined [here](./../../../sdk/python/feast/permissions/server).
22+
Few of the key models, classes to understand the authorization implementation on the client side can be found [here](./../../../sdk/python/feast/permissions/client).
23+
24+
## Configuring Authorization
25+
The authorization is configured using a dedicated `auth` section in the `feature_store.yaml` configuration.
26+
27+
**Note**: As a consequence, when deploying the Feast servers with the Helm [charts](../../../infra/charts/feast-feature-server/README.md),
28+
the `feature_store_yaml_base64` value must include the `auth` section to specify the authorization configuration.
29+
30+
### No Authorization
31+
This configuration applies the default `no_auth` authorization:
32+
```yaml
33+
project: my-project
34+
auth:
35+
type: no_auth
36+
...
37+
```
38+
39+
### OIDC Authorization
40+
With OIDC authorization, the Feast client proxies retrieve the JWT token from an OIDC server (or [Identity Provider](https://openid.net/developers/how-connect-works/))
41+
and append it in every request to a Feast server, using an [Authorization Bearer Token](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#bearer).
42+
43+
The server, in turn, uses the same OIDC server to validate the token and extract the user roles from the token itself.
44+
45+
Some assumptions are made in the OIDC server configuration:
46+
* The OIDC token refers to a client with roles matching the RBAC roles of the configured `Permission`s (*)
47+
* The roles are exposed in the access token passed to the server
48+
49+
(*) Please note that **the role match is case-sensitive**, e.g. the name of the role in the OIDC server and in the `Permission` configuration
50+
must be exactly the same.
51+
52+
For example, the access token for a client `app` of a user with `reader` role should have the following `resource_access` section:
53+
```json
54+
{
55+
"resource_access": {
56+
"app": {
57+
"roles": [
58+
"reader"
59+
]
60+
},
61+
}
62+
```
63+
64+
An example of OIDC authorization configuration is the following:
65+
```yaml
66+
project: my-project
67+
auth:
68+
type: oidc
69+
client_id: _CLIENT_ID__
70+
client_secret: _CLIENT_SECRET__
71+
realm: _REALM__
72+
auth_server_url: _OIDC_SERVER_URL_
73+
auth_discovery_url: _OIDC_SERVER_URL_/realms/master/.well-known/openid-configuration
74+
...
75+
```
76+
77+
In case of client configuration, the following settings must be added to specify the current user:
78+
```yaml
79+
auth:
80+
...
81+
username: _USERNAME_
82+
password: _PASSWORD_
83+
```
84+
85+
### Kubernetes RBAC Authorization
86+
With Kubernetes RBAC Authorization, the client uses the service account token as the authorizarion bearer token, and the
87+
server fetches the associated roles from the Kubernetes RBAC resources.
88+
89+
An example of Kubernetes RBAC authorization configuration is the following:
90+
{% hint style="info" %}
91+
**NOTE**: This configuration will only work if you deploy feast on Openshift or a Kubernetes platform.
92+
{% endhint %}
93+
```yaml
94+
project: my-project
95+
auth:
96+
type: kubernetes
97+
...
98+
```
99+
100+
In case the client cannot run on the same cluster as the servers, the client token can be injected using the `LOCAL_K8S_TOKEN`
101+
environment variable on the client side. The value must refer to the token of a service account created on the servers cluster
102+
and linked to the desired RBAC roles.

docs/getting-started/components/overview.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,3 +28,4 @@ A complete Feast deployment contains the following components:
2828
* **Batch Materialization Engine:** The [Batch Materialization Engine](batch-materialization-engine.md) component launches a process which loads data into the online store from the offline store. By default, Feast uses a local in-process engine implementation to materialize data. However, additional infrastructure can be used for a more scalable materialization process.
2929
* **Online Store:** The online store is a database that stores only the latest feature values for each entity. The online store is either populated through materialization jobs or through [stream ingestion](../../reference/data-sources/push.md).
3030
* **Offline Store:** The offline store persists batch data that has been ingested into Feast. This data is used for producing training datasets. For feature retrieval and materialization, Feast does not manage the offline store directly, but runs queries against it. However, offline stores can be configured to support writes if Feast configures logging functionality of served features.
31+
* **Authorization manager**: The authorization manager detects authentication tokens from client requests to Feast servers and uses this information to enforce permission policies on the requested services.

docs/getting-started/concepts/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,3 +31,7 @@
3131
{% content-ref url="dataset.md" %}
3232
[dataset.md](dataset.md)
3333
{% endcontent-ref %}
34+
35+
{% content-ref url="permission.md" %}
36+
[permission.md](permission.md)
37+
{% endcontent-ref %}

0 commit comments

Comments
 (0)