[Feature] Remove unused deps from workspace cargo.toml

### Component rpc ### Describe the feature you would like we have more deps than we need @yash-atreya ### Additional context _No response_