Skip to content

Add selective interception.#279

Merged
vbuberen merged 8 commits into
ChuckerTeam:developfrom
MiSikora:selective-interception
Mar 19, 2020
Merged

Add selective interception.#279
vbuberen merged 8 commits into
ChuckerTeam:developfrom
MiSikora:selective-interception

Conversation

@MiSikora

@MiSikora MiSikora commented Mar 18, 2020

Copy link
Copy Markdown
Contributor

📄 Context

There is an open issue that I declared to do - #266.

📝 Changes

I added to the public API a header that allows to skip interception. ChuckerInterceptor respects this header and does not process requests if ordered not to do so. It also strips out this header before passing it down the chain.

🚫 Breaking

Kind of, but not really. If someone in the past used a header like Skip-Chucker-Interceptor it would be no longer passed to the server. It would also skip Chucker processing if set to true.

🛠️ How to test

Unit tests are written. I don't think there is a point in adding anything to the HttpBinApi.

⏱️ Next steps

Closes #266

@vbuberen vbuberen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great feature. Thanks for suggesting and implementing.

Comment thread README.md Outdated
interceptor.redactHeader("Auth-Token", "User-Session");
```

### Skip-Inspection 🙅‍♀️🕵️

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: We have one emoji on each header. Maybe we should keep that uniform style?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to you :). I don't mind either way.

@vbuberen vbuberen merged commit d14ed64 into ChuckerTeam:develop Mar 19, 2020
Comment on lines +24 to +26
val request = chain.request().newBuilder()
.removeHeader(Chucker.SKIP_INTERCEPTOR_HEADER_NAME)
.build()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry if I came late to the game here. Just wanted to drop my opinion on the table.

I believe the library-no-op variant of the interceptor should not edit the request at all. I'm afraid this is going to create a precedent.

I really like the idea of having a Skip-Chucker-Interceptor special header, but it should be developer's duty to make sure it's applied only on debug requests and doesn't leak. We can provide a simple interceptor that will remove Chucker headers if on BuildConfig.DEBUG.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. But is there any point in stripping the header at all in this case? If header is not removed in the release builds it for sure does not matter if it is removed in debug and qa builds.

@MiSikora MiSikora Mar 21, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this would suggest that the stripping of header should be left to the consumers if they want to clean their requests before sending them to a server? An additional interceptor as a part of the library might be an option but I think it would have to be part of both library flavors.

We would just have to document to the users that this interceptor must be applied after ChuckerInterceptor for this feature to work.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or it could be a configuration part of the ChuckerInterceptor that defaults to false.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think we're having usability on one side vs security on the other. I do understand that the current implementation (that I really like btw) is the best from the developer point of view.

On the other hand this implementation leaked implementation details inside the library-no-op. By its nature, the -no-op library should do nothing at all. This is not the case anymore.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this would suggest that the stripping of header should be left to the consumers if they want to clean their requests before sending them to a server?

That could be an option absolutely.

An additional interceptor as a part of the library might be an option but I think it would have to be part of both library flavors.

Unfortunately yes.

@MiSikora MiSikora Mar 21, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was rather thinking about the user as they do not have to apply two interceptors this way but I totally get your point and will make a follow-up PR with an additional interceptor. I just want to clarify the point about header values first.

Comment on lines +14 to +15
const val SKIP_INTERCEPTOR_HEADER_NAME = "Skip-Chucker-Interceptor"
const val SKIP_INTERCEPTOR_HEADER = "$SKIP_INTERCEPTOR_HEADER_NAME: true"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the other comment, those two constants should be "" to make sure those don't leak in the Release APK

@MiSikora MiSikora Mar 21, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it would be the best user experience as users would have to be very mindful of how they use Chucker.

fun main() {
  try {
    Retrofit.Builder()
      .baseUrl("https://www.exmaple.com")
      .validateEagerly(true)
      .build()
      .create<Service>()
  } catch (e: Exception) {
    e.printStackTrace()
  }

  try {
    Request.Builder()
      .url("https://www.example.com")
      .addHeader("", "true")
      .build()
  } catch (e: Exception) {
    e.printStackTrace()
  }
}

interface Service {
  @GET("/") @Headers("") fun get()
}

This would result in exceptions.

java.lang.IllegalArgumentException: @Headers value must be in the form "Name: Value". Found: ""
    for method Service.get
	at retrofit2.Utils.methodError(Utils.java:53)
	at retrofit2.Utils.methodError(Utils.java:43)
	at retrofit2.RequestFactory$Builder.parseHeaders(RequestFactory.java:282)
	at retrofit2.RequestFactory$Builder.parseMethodAnnotation(RequestFactory.java:235)
	at retrofit2.RequestFactory$Builder.build(RequestFactory.java:171)
	at retrofit2.RequestFactory.parseAnnotations(RequestFactory.java:67)
	at retrofit2.ServiceMethod.parseAnnotations(ServiceMethod.java:26)
	at retrofit2.Retrofit.loadServiceMethod(Retrofit.java:192)
	at retrofit2.Retrofit.validateServiceInterface(Retrofit.java:179)
	at retrofit2.Retrofit.create(Retrofit.java:134)
	at MainKt.main(Main.kt:33)
	at MainKt.main(Main.kt)
java.lang.IllegalArgumentException: name is empty
	at okhttp3.Headers.checkName(Headers.java:269)
	at okhttp3.Headers$Builder.add(Headers.java:323)
	at okhttp3.Request$Builder.addHeader(Request.java:196)
	at MainKt.main(Main.kt:22)
	at MainKt.main(Main.kt)

@MiSikora MiSikora Mar 21, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing headers from the no-op library (and effectively from the core library) makes it a burden for the users to use this feature as they have to declare them in the project by themselves and need to keep track of implementation details changes.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it should be a developer's duty to make sure that you add the header if on Debug, and do not add it at all on release.

@MiSikora MiSikora Mar 21, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why though? It makes it really hard to use it with Retrofit as headers are added in annotations. Also not that pleasant with OkHttp if it is not Android module. I can see removing public header constants from both library flavors but not from one because it gets really confusing for the end users otherwise.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused to be honest. I also do not see a point in designing this around Retrofit. Solution with headers is designed around OkHttp and not Retrofit. Retrofit just allows to use it due to its configurability. It would look the same for any other network library that uses OkHttp. How the header is the delivered to that library is up to that library.

This feature couldn't be implemented for Retrofit with a CallAdapter. CallAdapters are called before application interceptors and they do not modify content of requests. The best they can do is to inspect or modify responses.

I don't feel discredited or anything. I just don't understand what should be done here and why :). From what I understood the expected implementation should be like this.

  • ChuckerInterceptor just reacts to the header but does not strip it.
  • Some other interceptor (i.e. ChuckerStrippingInterceptor) should be available in both library versions and deal with removing the header.
  • The header should not be public and just documented.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just don't understand what should be done here and why

Sorry, I was definitely not clear. Also I just wanted to drop my view here. I'm not asking for any change (given that this was already merged).

Given that the solution was implemented with custom headers, ChuckerInterceptor should behave in this way from my point of view:

  • On Debug: if the custom header is there, strip it, and skip the rest of the ChuckerInterceptor processing.
  • On Release: nothing, just pass over the request.

It's a developer's responsibility to make sure that the header is applied only on Debug build (i.e. use it at your own risk).

If the developer is using OkHTTP Request directly, this is trivial as:

if (BuildConfig.DEBUG) {
    build.addHeader(Chucker. SKIP_INTERCEPTOR_HEADER_NAME, "true")
}

If the developer is using Retrofit, this becomes not trivial as you can't selectively remove a Header when on Debug.

To solve this scenario we could either:

  • Provide a warning on the README about this situation.
  • Provide a sample interceptor in a snippet in the README that will strip out the extra header.
  • Provide a ChuckerStrippingInterceptor in a new module that will have to be applied as implementation dependency (both for debug and release), and should be applied after ChuckerInterceptor.

The header should not be public and just documented.

I'm not sure I follow you here.
The header should be a public field in the Debug version, and empty string in Release imho.

This feature couldn't be implemented for Retrofit with a CallAdapter.

Yup you're right. I was thinking about an alternative solution using custom annotations on the Retrofit side, but turns out is not that easy.

Anyway before we start coding, I'd love to get your point of view on this and find an agreement 👍 I don't want to impose my vision here.

@MiSikora MiSikora Mar 22, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so this is what I'm thinking. Personally, I don't see a problem with a no-op library doing cleaning but I totally understand your concerns about it and this is perfectly valid point.

As far as dealing with it I would just put a snippet and release a separate library for stripping headers. I have a library that would benefit from it as well so this would help there. I can make a release next week of this small stripping interceptor unless you want to have it as a part of ChuckerTeam. I don't mind either way, just don't want to duplicate libraries. :)

The point about headers is really not clear to me. I don't know why Chucker.SKIP_INTERCEPTOR_HEADER and Chucker.SKIP_INTERCEPTOR_HEADER_NAME should have empty values in the no-op flavor. While I understand that devs should be aware of what they apply and where, and I'm 100% with you on this, headers being empty strings is a bad thing in my opinion.

One thing is mentioned interaction with Retrofit (and most likely overwhelming amount of Chucker users are Retrofit users). Another thing is that is also not that easy to use with OkHttp. When I write my apps I put network layers in JVM modules which do not have access to BuildConfig. I could make same Gradle magic and generate this. I could inject information whether I'm in a debug mode or not. I could also inject an OkHttp client that does not have ChuckerInterceptor attached to it.

The best alternative is to define a top level static field that mimics what is in the library. But I would have to do this in every new project, be mindful of changes to this value and so on. It seems rather silly to me given that I just want not to use a QA tool for a specific network request. If I don't want this header to reach the server that's what stripping interceptor is for.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I understand that devs should be aware of what they apply and where, and I'm 100% with you on this, headers being empty strings is a bad thing in my opinion.

I see your points about passing empty headers + not being able to easily apply the header only on debug builds, and they're totally valid. My point was related to not having implementation details (Strings) bleeding in the -no-op library, but it makes sense to drop it in this case 👍 thanks for clarifying.

I can make a release next week of this small stripping interceptor unless you want to have it as a part of ChuckerTeam.

As for this, I'd love to get @vbuberen point of view as well.
I'm also happy to collect other feedbacks from the community here.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, joining the discussion here. I didn't think about concerns you have @cortinico and it seems that made a a decision too fast about merging this PR. My suggestion is to revert this change for 3.2.0 and return it when we have a better implementation.

As to additional interceptor suggested by @MiSikora - in case we are with this solution I would prefer to have it in ChuckerTeam so it will have more credibility for users.

@vbuberen vbuberen mentioned this pull request Apr 2, 2020
cortinico added a commit that referenced this pull request Apr 4, 2020
cortinico added a commit that referenced this pull request Apr 4, 2020
vbuberen added a commit that referenced this pull request Apr 4, 2020
* Remove scroll flags (#210)

* Fix/gradle properties (#211)

* Allow Gradle parallel build
* Fix version name

* Fix for curl command (#214)

* Fix R8 minification crash on TransactionViewModel creation. (#219)

* Big resources renaming (#216)

* Fix clear action crash when application is dead (#222)

* Fix for crash on Save transaction action (#221)

* Show warning is transaction is null, fix crash in Save action

* Uncomment sample transactions

* Replace mulptiple returning with multiple throw due to detekt issue

* Add message into IOException, update string for request being not ready

* Fix for NPE in NotificationHelper (#223)

* Add additional check fo transaction being not null before getting its notificationText

* Extract transaction item from transactionBuffer

* ViewModel refactoring (#220)

* Update ViewModel dependency, refactor TransactionViewModel
* Dependencies clean up
* Switch to ViewModel on the main screen

* Fix depleting bytes from the response. (#226)

* Use HttpUrl instead of Uri for parsing URL data.
* Do not read image sources directly from the response.
* Simplify gzip logic.
* Move gzip checks from IoUtils to OkHttpUtils.
* Remove unused 'Response.hasBody()' extension.
* Update library/src/main/java/com/chuckerteam/chucker/internal/support/OkHttpUtils.kt

* Revert resource renaming (#227)

* Revert renaming

* Add changelgos for 3.1.2 (#230)

* Add missing  section to release 3.1.1 and 3.1.2 (#232)

* Update Github templates (#235)

* Update templates
* Remove redundant dot
* Remove default `no` from the checkbox

* Switch to platform defined HTTP code constants (#238)

* Add instruction for checkbox selection (#237)

* Fix HttpHeader serialization when obfuscation is enabled (#240)

* Update README (#243)

* Add Action to validate the Gradle Wrapper (#245)

* Gradle to 6.2 (#247)

* Do not refresh transaction when it is not being affected. (#246)

* Do not refresh transaction when it is not being affected.
* Use correct null-aware comparison for HttpTransaction.

* Add switching between encoded and decoded URL formats. (#233)

* Add switching between encoded and decoded URL formats.
* Make URL encoding transaction specific.
* Change test name for upcoming #244 PR.
* Use LiveDataRecord for combineLatest tests.
* Properly switch encoding and decoding URL.
* Show encoding icon only when it is viable.
* Add encoded URL samples to HttpBinClient.
* Avoid using 'this' scoping mechanism for URL.

* Fix typo in feature request comment (#251)

* RTL Support (#208)

* Remove ltr forcing and replace ScrollView in Overview
* Replace Overview layout, add rtl support for it
* Add textDirection and textAlignment property for API 21+
* Fix host textview constraints
* Replace android:src with app:srcCompat
* Update ids for layouts to avoid clashes
* Update Material components to stable
* Remove supportsRTL tag from Manifest, update Gradle plugin
* Styles update
* Remove supportsRTL from library manifest
* Revert usage of supportVectorDrawables to avoid crashes on APIs 16-19 due to notifications icons
* Fix lint issue with vector drawable

* Response search fixes (#256)

* Fix response search to be case insensitive
* Add minimum required symbols
* Fix invalid options menu in response fragment

* Feature/tls info (#252)

* Add UI for TLS info

* Implement logic for retrieving TLS info

* Address code review feedback

* Switch views to ViewBinding (#253)

* Switch to ViewBinding in activities
* Switch to ViewBinding in ErrorsAdapter, add formattedDate field into Throwable models
* Transaction adapter switch to ViewBinding
* Remove variable for formatted date from models
* Switch to ViewBinding in TransactionPayloadAdapter
* Switch to ViewBinding in TransactionPaayload and TransactionOverviewFragments
* Switch list fragments to ViewBinding
* Fix link for tutorial opening
* Rename views
* Address code review feedback
* Hide SSL field if isSSL is null

* Libs update (#260)

* Update tools versions
* JUnit update

* Feature/truth (#258)

* Add Truth, update part of test classes
* Convert other tests to use Truth, fix date parser test
* Add Truth assertions to FormatUtilsTest, fix ktlint issue
* Update assertions to a proper syntax

* Add missing ThrowableViewModel (#257)

* Add Error ViewModel, update title in TransactionActivity in onCreate
* Switch from errors to throwable naming to have a uniform naming style
* Rename toolbar title

* Migrating from Travis to GH Actions (#262)

* Setup GH Actions

* Run only on Linux

* Remove Travis File

* Run only gradlew directly

* Update targetSDK and Kotlin (#264)

* Add stale builds canceller (#265)

* Add filters
* Update Gradle wrapper validation workflow
* Update pre-merge workflow

* Fixed various Lints (#268)

* fixed typos
* fixed KDocs

* Replace Travis badge with GH Actions badge (#269)

* Remove redundant JvmName (#274)

* Fix margins and paddings for payload items (#277)

* Add selective interception. (#279)

* Add selective interception.
* Update README.md.
* Align formatting in README with other points.
* Avoid header name duplication.
* Strip interception header also in the no-op library.

* UX improvements (#275)

* Add icon for non-https transactions
* Update secondary color to be more contrast
* Simplify protocol resources setting

* Add tests to format utils (#281)

* add tests to format utils

* fixes after code review

* formatting fix

Co-authored-by: adammasyk <adam.masyk@programisci.eu>

* format utils test refactor (#285)

* format utils test refactor
* share text test refactor

* Migrate to Kotlin Coroutines (#270)

* Add coroutine as a dependency in build.gradle
* Migrate AsyncTasks to kotlin coroutines
* Migrate executors with the coroutines in repositories

* Multi cast upstream response for Chucker consumption. (#267)

* Multi cast response upstream for Chucker consumption.

* Read buffer prefix before potentially gunzipping it.

* Inform Chucker about unprocessed responses.

* Simplify multi casting logic.

* Move read offset to a variable.

* Inline one-line method.

* Give better control over TeeSource read results.

* Add documentation to TeeSource.

* Close side channel when capacity is exceeded.

Co-authored-by: Volodymyr Buberenko <vbuberen@users.noreply.github.com>

* Remove unnecessary mock method. (#289)

* removed redundant Gson configurations (#290)

* increased test coverage for format utils (#291)

Co-authored-by: Karthik R <karthr@paypal.com>

* added few test cases for json formatting (#295)

* Properly handle unexhausted network responses (#288)

* Handle properly not consumed upstream body.
* Handle IO issues while reading from file.

* Update dependencies (#296)

* Update depencies

* Update OkHttp to 3.12.10

* Handle empty and missing response bodies. (#250)

* Add failing test cases.
* Remove unused const.
* Gzip response body if it was gunzipped.
* Add test cases for regular bodies in Chucker.
* Fix rule formatting.
* Use proper name for application interceptor.
* Return original response downstream.
* Account for no content with gzip encoding.
* Use Truth for Chucker tests.
* Honor empty plaintext bodies.
* Revert changes to HttpBinClient.
* Update library/src/test/java/com/chuckerteam/chucker/ChuckerInterceptorDelegate.kt
* Update library/src/main/java/com/chuckerteam/chucker/internal/support/OkHttpUtils.kt
Co-authored-by: Nicola Corti <corti.nico@gmail.com>
Co-authored-by: Volodymyr Buberenko <vbuberen@users.noreply.github.com>

* Add hasFixed size to RecyclerViews (#297)

* Detekt to 1.7.4 (#299)

* Revert "Add selective interception. (#279)" (#300)

This reverts commit d14ed64.

* Prepare 3.2.0 (#298)

* Update versions info

* Update Changelog

* Fix links and update date

Co-authored-by: Michał Sikora <michalsikora90@gmail.com>
Co-authored-by: Nicola Corti <corti.nico@gmail.com>
Co-authored-by: Sergey Chelombitko <119192+technoir42@users.noreply.github.com>
Co-authored-by: Michał Sikora <me@mehow.io>
Co-authored-by: Hitanshu Dhawan <hitanshudhawan1996@gmail.com>
Co-authored-by: adammasyk <masiol91@gmail.com>
Co-authored-by: adammasyk <adam.masyk@programisci.eu>
Co-authored-by: Nikhil Chaudhari <nikhyl777@gmail.com>
Co-authored-by: karthik rk <karthik.rk18@gmail.com>
Co-authored-by: Karthik R <karthr@paypal.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow to selectively skip Chucker interception

3 participants