Skip to content

Feature: --no-cache takes a regexp to ignore matching lines in Dockerfile#4322

Closed
timruffles wants to merge 1 commit intomoby:masterfrom
timruffles:1996-no-cache-regexp
Closed

Feature: --no-cache takes a regexp to ignore matching lines in Dockerfile#4322
timruffles wants to merge 1 commit intomoby:masterfrom
timruffles:1996-no-cache-regexp

Conversation

@timruffles
Copy link
Contributor

  • modifies CLI and server to accept a regexp for no-cache
  • adds defaultable flags to mflag to support existing bool no-cache
  • applies to matching RUN and ADD lines

Closes #1996

@timruffles
Copy link
Contributor Author

pull request for #1996

@SvenDowideit
Copy link
Contributor

I fear this deserves an example, also, which regexp's are you supporting? ( http://en.wikipedia.org/wiki/Regular_expression#Standards )

@timruffles
Copy link
Contributor Author

@SvenDowideit good points. Added an example, and specified that it's a golang regexp

@SvenDowideit
Copy link
Contributor

oh gosh - for an awful lot of people saying 'golang regexp' will make them groan, both as it implies that golang hasn't implemented one of the standard regexp syntax's, and worse, isn't used by enough people that use the horrid corners of regexp.

Going off and reading the regexp package docs suggests to me that it might be a good idea to link to https://code.google.com/p/re2/wiki/Syntax - as that gives a good over-view of what is, and isn't supported.

@unclejack
Copy link
Contributor

As @SvenDowideit has stated above, we can't simply tell people that it's a 'golang regexp'. This also needs some examples, in addition to a link to the regexp syntax documentation.

@timruffles
Copy link
Contributor Author

I'll add some extra docs and examples.

As a thought - would limiting it to POSIX regexp be a good idea? It's a better known syntax, has a smaller API surface area, and is less likely to change than RE2. The use case is normally '--no-cache "git clone"' so POSIX regexp should be plenty to cover it.

@SvenDowideit
Copy link
Contributor

I prefer standards - it makes it easier for us to communicate :)

@timruffles
Copy link
Contributor Author

@SvenDowideit agreed - I'd feel more confident to see POSIX in a tool than almost-RE2 :)

Changed to use POSIX ERE throughout, updated docs, and added an example.

@SvenDowideit
Copy link
Contributor

Docs LGTM

1 similar comment
@jamtur01
Copy link
Contributor

Docs LGTM

@unclejack
Copy link
Contributor

I'm seeing a failure in the tests:

[error] server.go:1221 No such container: id_not_found
--- FAIL: TestBuildImageWithoutCache (0.11 seconds)
        buildfile_test.go:581: Cache misbehavior, got hit=true, expected hit=false: (first: fe910d1bb1bb0d177234c68c16dd72c8958e949ae79c27bd6e78a1244685755d, second fe910d1bb1bb0d177234c68c16dd72c8958e949ae79c27bd6e78a1244685755d)

@timruffles
Copy link
Contributor Author

@unclejack sorry for slow reply - tests were passing for me, still passing now (rebased all new work).

api/client.go Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we make this a generic RegexFlag? What do you think? I also think this struct can be moved into the /opts package.

@crosbymichael
Copy link
Contributor

I'm getting a build error

root@9509f414ea95:/go/src/github.com/dotcloud/docker# ./hack/make.sh binary test test-integration

---> Making bundle: binary (in bundles/0.9.1-dev/binary)
# github.com/dotcloud/docker/server
server/server.go:29: runtime redeclared as imported package name
        previous declaration at server/server.go:16
server/server.go:2458: undefined: runtime.Runtime

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can group and simplify this with

var (
    useCacheRe *regexp.Regexp
    ignoreCacheRe = regexp.MustCompilePOSIX(".*")
)

@crosbymichael
Copy link
Contributor

@vieux Can you take a look at the mflag changes? Maybe there is a better way to handle this.

@vieux
Copy link
Contributor

vieux commented May 1, 2014

flags look ok, @shykes what do you think about the feature

@timruffles
Copy link
Contributor Author

Updated with approach @tianon suggested in #1996. It adds a new --break-cache flag, rather than changing --no-cache (would break scripts that'd specified true or false). This is simpler, and didn't need additions to mflag supporting boolean/string flags.

The PR is rough and non-working after these changes - can't test as can't pull images while accessing internet via tethering. Wanted to check the approach & docs (cc @SvenDowideit) are acceptable before continuing!

…file

- modifies CLI and server to accept a regexp for no-cache
- adds defaultable flags to mflag to support existing bool no-cache
- applies to matching RUN and ADD lines

Docker-DCO-1.1-Signed-off-by: Tim Ruffles <oi@truffles.me.uk> (github: timruffles)
@shykes
Copy link
Contributor

shykes commented May 2, 2014

I am a +1 on this feature, we discussed it a few times.

I will gladly take it, however as a general rule please don't assign PRs to others - they should be self-assigned, otherwise I worry that we will start mentally labelling it as spam and ignoring assignment.

@vieux vieux assigned vieux and unassigned shykes Jul 9, 2014
@crosbymichael
Copy link
Contributor

@vieux are you going to carry this PR or do you want me to?

@vieux
Copy link
Contributor

vieux commented Jul 30, 2014

@timruffles do you have time to work on this ?

@timruffles
Copy link
Contributor Author

Sure - I guess it just needs merging?

On Thursday, 31 July 2014, Victor Vieux <notifications@github.com
javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

@timruffles https://github.com/timruffles do you have time to work on
this ?


Reply to this email directly or view it on GitHub
#4322 (comment).

@vieux
Copy link
Contributor

vieux commented Jul 31, 2014

@timruffles you last commit is completely broken. I guess you somehow messed up when you pushed.
Even with a rebase it doesn't compile.

For example https://github.com/docker/docker/pull/4322/files#diff-e728bf75e7a6536ac0e70b85f7aaf65aR111

It's a cmd.String but the default value is false

@timruffles
Copy link
Contributor Author

Yes, sorry - should have read my own comment. I pushed stuff I couldn't compile/build as my dev env was all screwed up and I had no internet.

Looks like quite a big merge job as lots of files have moved. I'm working on it now, and having great fun with boot2docker not working.

@tiborvass
Copy link
Contributor

@timruffles I'm closing this for now. Feel free to update your branch and comment below we will gladly reopen it.

@cancan101
Copy link

What ever became of this idea?

@nathanjsweet
Copy link

This proposal will solve this issue, and in much more elegant way, IMO. If what folks are looking for is a way to invalidate cache when something like a git repo actually changes you could take the following approach:

$>docker build --build-arg git_commit=$(git rev-parse HEAD) ...
//the file:
FROM busybox
RUN git clone https://git.foo.com/myrepo.git
ARG git_commit
RUN git fetch https://git.foo.com/myrepo.git $git_commit
RUN git merge $git_commit

The proposed patch has been merged or approved, I believe, so hopefully we'll see it out soon.

@duglin
Copy link
Contributor

duglin commented Oct 7, 2015

in the more generic case you can also do:

$ docker build --build-arg bust=$(date) ...

FROM busybox
...
RUN echo $bust
...

will never cache from that RUN line thru the rest of the Dockerfile. Just too bad that people need the hack in the first place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New feature request: Selectively disable caching for specific RUN commands in Dockerfile