Skip to content
This repository has been archived by the owner on Sep 5, 2019. It is now read-only.

Build reported as a timeout when it actually succeeded #599

Open
scothis opened this issue Apr 17, 2019 · 0 comments · Fixed by tektoncd/pipeline#801
Open

Build reported as a timeout when it actually succeeded #599

scothis opened this issue Apr 17, 2019 · 0 comments · Fixed by tektoncd/pipeline#801
Labels
kind/bug Something isn't working

Comments

@scothis
Copy link
Contributor

scothis commented Apr 17, 2019

Expected Behavior

Successful builds completed before the timeout are reported as successful.

Actual Behavior

Successful builds can be reported as a timeout.

In this case, the build and pod are created at 2019-04-17T21:02:21Z, and finish 31 seconds later at 2019-04-17T21:02:52Z. Instead of the build's Succeeded condition reporting True the message is Build "square-build" failed to finish within "10m0s".

build

apiVersion: build.knative.dev/v1alpha1
kind: Build
metadata:
  creationTimestamp: "2019-04-17T21:02:21Z"
  generation: 1
  labels:
    projectriff.io/function: square
  name: square-build
  namespace: default
  resourceVersion: "2531684"
  selfLink: /apis/build.knative.dev/v1alpha1/namespaces/default/builds/square-build
  uid: 17a0e49f-6154-11e9-9a6b-42010a80016c
spec:
  Status: ""
  serviceAccountName: riff-build
  source:
    git:
      revision: master
      url: https://github.com/projectriff/fats.git
    subPath: functions/uppercase/node
  template:
    arguments:
    - name: IMAGE
      value: gcr.io/cf-sandbox-sandrews/square
    - name: CACHE
      value: cache
    kind: ClusterBuildTemplate
    name: riff-cnb
  timeout: 10m0s
  volumes:
  - name: cache
    persistentVolumeClaim:
      claimName: build-cache-square
status:
  builder: Cluster
  cluster:
    namespace: default
    podName: square-build-pod-4cad55
  completionTime: "2019-04-17T21:02:54Z"
  conditions:
  - lastTransitionTime: "2019-04-17T21:02:54Z"
    message: Build "square-build" failed to finish within "10m0s"
    reason: BuildTimeout
    status: "False"
    type: Succeeded
  startTime: "2019-04-17T20:52:54Z"
  stepsCompleted: null

pod (sorry there are so many steps)

apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubernetes.io/limit-ranger: 'LimitRanger plugin set: cpu request for container
      nop; cpu request for init container build-step-credential-initializer; cpu request
      for init container build-step-git-source-0; cpu request for init container build-step-prepare;
      cpu request for init container build-step-detect; cpu request for init container
      build-step-analyze; cpu request for init container build-step-build; cpu request
      for init container build-step-export'
    sidecar.istio.io/inject: "false"
  creationTimestamp: "2019-04-17T21:02:21Z"
  labels:
    build.knative.dev/buildName: square-build
    projectriff.io/function: square
  name: square-build-pod-6070c4
  namespace: default
  ownerReferences:
  - apiVersion: build.knative.dev/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: Build
    name: square-build
    uid: 17a0e49f-6154-11e9-9a6b-42010a80016c
  resourceVersion: "2531685"
  selfLink: /api/v1/namespaces/default/pods/square-build-pod-6070c4
  uid: 17b22b73-6154-11e9-9a6b-42010a80016c
spec:
  containers:
  - image: gcr.io/knative-releases/github.com/knative/build/cmd/nop@sha256:a318ee728d516ff732e2861c02ddf86197e52c6288049695781acb7710c841d4
    imagePullPolicy: IfNotPresent
    name: nop
    resources:
      requests:
        cpu: 100m
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: riff-build-token-5p282
      readOnly: true
  dnsPolicy: ClusterFirst
  initContainers:
  - args:
    - -basic-docker=push-credentials=https://gcr.io
    env:
    - name: HOME
      value: /builder/home
    image: gcr.io/knative-releases/github.com/knative/build/cmd/creds-init@sha256:ebf58f848c65c50a7158a155db7e0384c3430221564c4bbaf83e8fbde8f756fe
    imagePullPolicy: IfNotPresent
    name: build-step-credential-initializer
    resources:
      requests:
        cpu: 100m
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /workspace
      name: workspace
    - mountPath: /builder/home
      name: home
    - mountPath: /var/build-secrets/push-credentials
      name: secret-volume-push-credentials
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: riff-build-token-5p282
      readOnly: true
    workingDir: /workspace
  - args:
    - -url
    - https://github.com/projectriff/fats.git
    - -revision
    - master
    env:
    - name: HOME
      value: /builder/home
    image: gcr.io/knative-releases/github.com/knative/build/cmd/git-init@sha256:09f22919256ba4f7451e4e595227fb852b0a55e5e1e4694cb7df5ba0ad742b23
    imagePullPolicy: IfNotPresent
    name: build-step-git-source-0
    resources:
      requests:
        cpu: 100m
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /workspace
      name: workspace
    - mountPath: /builder/home
      name: home
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: riff-build-token-5p282
      readOnly: true
    workingDir: /workspace
  - args:
    - -c
    - |
      chown -R "1000:1000" "/builder/home" && chown -R "1000:1000" /layers && chown -R "1000:1000" /workspace
    command:
    - /bin/sh
    env:
    - name: HOME
      value: /builder/home
    image: ubuntu:18.04
    imagePullPolicy: Always
    name: build-step-prepare
    resources:
      requests:
        cpu: 100m
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /layers
      name: cache
    - mountPath: /workspace
      name: workspace
      subPath: functions/uppercase/node
    - mountPath: /builder/home
      name: home
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: riff-build-token-5p282
      readOnly: true
    workingDir: /workspace
  - args:
    - -app=/workspace
    - -group=/layers/group.toml
    - -plan=/layers/plan.toml
    command:
    - /lifecycle/detector
    env:
    - name: HOME
      value: /builder/home
    - name: RIFF
      value: "true"
    - name: RIFF_ARTIFACT
    - name: RIFF_HANDLER
    - name: RIFF_OVERRIDE
    image: projectriff/builder:0.2.0
    imagePullPolicy: Always
    name: build-step-detect
    resources:
      requests:
        cpu: 100m
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /layers
      name: cache
    - mountPath: /workspace
      name: workspace
      subPath: functions/uppercase/node
    - mountPath: /builder/home
      name: home
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: riff-build-token-5p282
      readOnly: true
    workingDir: /workspace
  - args:
    - -layers=/layers
    - -helpers=true
    - -group=/layers/group.toml
    - gcr.io/cf-sandbox-sandrews/square
    command:
    - /lifecycle/analyzer
    env:
    - name: HOME
      value: /builder/home
    image: projectriff/builder:0.2.0
    imagePullPolicy: Always
    name: build-step-analyze
    resources:
      requests:
        cpu: 100m
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /layers
      name: cache
    - mountPath: /workspace
      name: workspace
      subPath: functions/uppercase/node
    - mountPath: /builder/home
      name: home
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: riff-build-token-5p282
      readOnly: true
    workingDir: /workspace
  - args:
    - -layers=/layers
    - -app=/workspace
    - -group=/layers/group.toml
    - -plan=/layers/plan.toml
    command:
    - /lifecycle/builder
    env:
    - name: HOME
      value: /builder/home
    - name: RIFF
      value: "true"
    - name: RIFF_ARTIFACT
    - name: RIFF_HANDLER
    - name: RIFF_OVERRIDE
    image: projectriff/builder:0.2.0
    imagePullPolicy: Always
    name: build-step-build
    resources:
      requests:
        cpu: 100m
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /layers
      name: cache
    - mountPath: /workspace
      name: workspace
      subPath: functions/uppercase/node
    - mountPath: /builder/home
      name: home
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: riff-build-token-5p282
      readOnly: true
    workingDir: /workspace
  - args:
    - -layers=/layers
    - -helpers=true
    - -app=/workspace
    - -image=packs/run:0.1.0
    - -group=/layers/group.toml
    - gcr.io/cf-sandbox-sandrews/square
    command:
    - /lifecycle/exporter
    env:
    - name: HOME
      value: /builder/home
    image: projectriff/builder:0.2.0
    imagePullPolicy: Always
    name: build-step-export
    resources:
      requests:
        cpu: 100m
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /layers
      name: cache
    - mountPath: /workspace
      name: workspace
      subPath: functions/uppercase/node
    - mountPath: /builder/home
      name: home
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: riff-build-token-5p282
      readOnly: true
    workingDir: /workspace
  nodeName: gke-riff-default-pool-f902d6b2-qps0
  priority: 0
  restartPolicy: Never
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: riff-build
  serviceAccountName: riff-build
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: cache
    persistentVolumeClaim:
      claimName: build-cache-square
  - emptyDir: {}
    name: empty-dir
  - emptyDir: {}
    name: workspace
  - emptyDir: {}
    name: home
  - name: secret-volume-push-credentials
    secret:
      defaultMode: 420
      secretName: push-credentials
  - name: riff-build-token-5p282
    secret:
      defaultMode: 420
      secretName: riff-build-token-5p282
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2019-04-17T21:02:53Z"
    reason: PodCompleted
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2019-04-17T21:02:23Z"
    reason: PodCompleted
    status: "False"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2019-04-17T21:02:23Z"
    reason: PodCompleted
    status: "False"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2019-04-17T21:02:23Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: docker://8bc79cc6b1c2f06ab84de9dd379f463d51b0310840c9f55ca0413a146d038e44
    image: sha256:94cec99fc35341544691c5501ba27d0ed7b659a1e7026c9b8cd2f6d6c7dfb310
    imageID: docker-pullable://gcr.io/knative-releases/github.com/knative/build/cmd/nop@sha256:a318ee728d516ff732e2861c02ddf86197e52c6288049695781acb7710c841d4
    lastState: {}
    name: nop
    ready: false
    restartCount: 0
    state:
      terminated:
        containerID: docker://8bc79cc6b1c2f06ab84de9dd379f463d51b0310840c9f55ca0413a146d038e44
        exitCode: 0
        finishedAt: "2019-04-17T21:02:53Z"
        reason: Completed
        startedAt: "2019-04-17T21:02:53Z"
  hostIP: 10.128.0.12
  initContainerStatuses:
  - containerID: docker://793a3b89ec971a7f2edc44e01d27c137e56f5a03026631114554a3689fdc3264
    image: sha256:ea5d3f4e41798996e3962d6f2398f508e83db313b14574128e5ec0699abbcbc8
    imageID: docker-pullable://gcr.io/knative-releases/github.com/knative/build/cmd/creds-init@sha256:ebf58f848c65c50a7158a155db7e0384c3430221564c4bbaf83e8fbde8f756fe
    lastState: {}
    name: build-step-credential-initializer
    ready: true
    restartCount: 0
    state:
      terminated:
        containerID: docker://793a3b89ec971a7f2edc44e01d27c137e56f5a03026631114554a3689fdc3264
        exitCode: 0
        finishedAt: "2019-04-17T21:02:34Z"
        reason: Completed
        startedAt: "2019-04-17T21:02:34Z"
  - containerID: docker://b66538c49e9f0358e2aec006f5678d7cb295141276e74658ea3ad890ae8e3c2b
    image: sha256:97f4e092c75a3f15e258b74b3c1b9fb9cb5fb3bb087a58da0452cd3eb3dea2a6
    imageID: docker-pullable://gcr.io/knative-releases/github.com/knative/build/cmd/git-init@sha256:09f22919256ba4f7451e4e595227fb852b0a55e5e1e4694cb7df5ba0ad742b23
    lastState: {}
    name: build-step-git-source-0
    ready: true
    restartCount: 0
    state:
      terminated:
        containerID: docker://b66538c49e9f0358e2aec006f5678d7cb295141276e74658ea3ad890ae8e3c2b
        exitCode: 0
        finishedAt: "2019-04-17T21:02:35Z"
        reason: Completed
        startedAt: "2019-04-17T21:02:34Z"
  - containerID: docker://7fe03c572cc965207c6ee0ecd915fb0924d17f0d670a8d65135c0ec5d83a1205
    image: ubuntu:18.04
    imageID: docker-pullable://ubuntu@sha256:f2557f94cac1cc4509d0483cb6e302da841ecd6f82eb2e91dc7ba6cfd0c580ab
    lastState: {}
    name: build-step-prepare
    ready: true
    restartCount: 0
    state:
      terminated:
        containerID: docker://7fe03c572cc965207c6ee0ecd915fb0924d17f0d670a8d65135c0ec5d83a1205
        exitCode: 0
        finishedAt: "2019-04-17T21:02:35Z"
        reason: Completed
        startedAt: "2019-04-17T21:02:35Z"
  - containerID: docker://52d54ef79aeb4d294d5470f6983b4f25a2e18702fb45346b9f2619efed595b18
    image: projectriff/builder:0.2.0
    imageID: docker-pullable://projectriff/builder@sha256:dccac174b2e80186aab20ed3a52a61e8a69d6c023e16f45c53c6385d7f5b5c4c
    lastState: {}
    name: build-step-detect
    ready: true
    restartCount: 0
    state:
      terminated:
        containerID: docker://52d54ef79aeb4d294d5470f6983b4f25a2e18702fb45346b9f2619efed595b18
        exitCode: 0
        finishedAt: "2019-04-17T21:02:37Z"
        reason: Completed
        startedAt: "2019-04-17T21:02:37Z"
  - containerID: docker://a98ce05337738a7c633acf2ca65c9902542359ac038e745af0dcef8e1d812c5b
    image: projectriff/builder:0.2.0
    imageID: docker-pullable://projectriff/builder@sha256:dccac174b2e80186aab20ed3a52a61e8a69d6c023e16f45c53c6385d7f5b5c4c
    lastState: {}
    name: build-step-analyze
    ready: true
    restartCount: 0
    state:
      terminated:
        containerID: docker://a98ce05337738a7c633acf2ca65c9902542359ac038e745af0dcef8e1d812c5b
        exitCode: 0
        finishedAt: "2019-04-17T21:02:38Z"
        reason: Completed
        startedAt: "2019-04-17T21:02:38Z"
  - containerID: docker://3bc87df69cb5166f261fcaa6a3f8ec7bbb7504da2e62066491c535bdabb53b05
    image: projectriff/builder:0.2.0
    imageID: docker-pullable://projectriff/builder@sha256:dccac174b2e80186aab20ed3a52a61e8a69d6c023e16f45c53c6385d7f5b5c4c
    lastState: {}
    name: build-step-build
    ready: true
    restartCount: 0
    state:
      terminated:
        containerID: docker://3bc87df69cb5166f261fcaa6a3f8ec7bbb7504da2e62066491c535bdabb53b05
        exitCode: 0
        finishedAt: "2019-04-17T21:02:46Z"
        reason: Completed
        startedAt: "2019-04-17T21:02:40Z"
  - containerID: docker://73aecaac4b79a7438e079b6b4d90e7585aadc31521503e6a8731f6443c248e26
    image: projectriff/builder:0.2.0
    imageID: docker-pullable://projectriff/builder@sha256:dccac174b2e80186aab20ed3a52a61e8a69d6c023e16f45c53c6385d7f5b5c4c
    lastState: {}
    name: build-step-export
    ready: true
    restartCount: 0
    state:
      terminated:
        containerID: docker://73aecaac4b79a7438e079b6b4d90e7585aadc31521503e6a8731f6443c248e26
        exitCode: 0
        finishedAt: "2019-04-17T21:02:52Z"
        reason: Completed
        startedAt: "2019-04-17T21:02:47Z"
  phase: Succeeded
  podIP: 10.48.0.92
  qosClass: Burstable
  startTime: "2019-04-17T21:02:23Z"

Steps to Reproduce the Problem

  1. create a build
  2. ...?
  3. see that the build is reported as a timeout, when it actually succeeded

I don't have steps to reproduce at this moment, but wanted to capture the issue in case others see similar behavior.

Additional Info

Knative build 0.5.0, GKE v1.12.6-gke.10
@knative-prow-robot knative-prow-robot added the kind/bug Something isn't working label Apr 17, 2019
This was referenced Apr 25, 2019
Closed
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
kind/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix build uncorrect status ustackq/pipeline
fix build uncorrect status ustackq/build
2 participants
@scothis @knative-prow-robot