From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7BEDC4708F for ; Tue, 1 Jun 2021 22:44:20 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7982A613BC for ; Tue, 1 Jun 2021 22:44:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7982A613BC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 071C56EB27; Tue, 1 Jun 2021 22:44:20 +0000 (UTC) Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by gabe.freedesktop.org (Postfix) with ESMTPS id 25F806EB2A; Tue, 1 Jun 2021 22:44:19 +0000 (UTC) Received: by mail-pj1-x102c.google.com with SMTP id m13-20020a17090b068db02901656cc93a75so2301747pjz.3; Tue, 01 Jun 2021 15:44:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f8TBvuEhJIYSx02uYrz2RNVyn3Huf1vKVuYXCtSr4og=; b=k/PB6bvkWkvSgogGIYMhCtAdJ0mvCmfJfEVvf7QncXwzBg2k2fk14ifPK0KgHJNLos kzvHtPyuApp5iVxnADmD41EMosbEUGtYga2eRNuKGSl7BIAY74/0uXIsYpdQjIInJ3q+ ss1qcRZD6NoHfwG1pqgH0A5z0AGn8f6xNWLTVggnuirneqbKvHAqTJCXQqtgsyZt4pgB +5m8T309VhmK7VEFXUw13i+jt4XbpbpqPlTPgRQvCGx0YpojKySY3Xv0lLXJJYVNEBSu JweZoy/ZzlvR70cEWD8d8iGrZVF8fnWEaeQxPbxvM9M/FObwTKZezXhC64u/2Or9CSkh EpIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f8TBvuEhJIYSx02uYrz2RNVyn3Huf1vKVuYXCtSr4og=; b=AMcq3HoNQcYt+7GZMWUDydz+4gQZEuxdwLvt1zNCl+h6q8854tjTkruCiUuwvAYv99 eWTo2FWBxZoGBSWohfiCZVrjHz/E8DeORICxteDd5RrChRvp+lGvvmKAyo1E7Dj+5Ueg FrUzbCXOS50QJZIk8fyel1Y9d77HpBJDPtXLYffck24Bzvy1vYibo693Cb3DB+PRlIoN zvyyAjWG+RJC0J8yhGw/73ERb5yNiaECom5sQkW0gVqGs4SxtNDKHmVQhyZIDUWo14oU TLFiCqsinwCLh+g9mK/QezcUpZOG1H+TR1h/NE2SUxZCCowC+qPWLHFlh3hLx2rquRrH mdnQ== X-Gm-Message-State: AOAM531rzQPkeSR6KozN+sMyS6+VnNIFwhOcrjepZsqskerk+EDkS/3b zBdCAQvTkGGY0o4gKBX/ITo35aFf6ZqEW2Qk X-Google-Smtp-Source: ABdhPJzHoeK4rxw7lnJietI1BhJLhslhGDKNcFVd62Bh/uRAUklK4pKybYSgWYKXeqMUpca3bydJOA== X-Received: by 2002:a17:902:e551:b029:103:c082:ba with SMTP id n17-20020a170902e551b0290103c08200bamr14935051plf.3.1622587458135; Tue, 01 Jun 2021 15:44:18 -0700 (PDT) Received: from localhost (c-73-25-156-94.hsd1.or.comcast.net. [73.25.156.94]) by smtp.gmail.com with ESMTPSA id s123sm13990015pfb.78.2021.06.01.15.44.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Jun 2021 15:44:17 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Subject: [PATCH v4 3/6] drm/msm: Improve the a6xx page fault handler Date: Tue, 1 Jun 2021 15:47:22 -0700 Message-Id: <20210601224750.513996-5-robdclark@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210601224750.513996-1-robdclark@gmail.com> References: <20210601224750.513996-1-robdclark@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Rob Clark , "open list:DRM DRIVER FOR MSM ADRENO GPU" , Sai Prakash Ranjan , Douglas Anderson , Akhil P Oommen , Jonathan Marek , David Airlie , "open list:DRM DRIVER FOR MSM ADRENO GPU" , Sharat Masetty , Konrad Dybcio , Jordan Crouse , Jordan Crouse , "Kristian H. Kristensen" , AngeloGioacchino Del Regno , Marijn Suijten , Sean Paul , open list Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Jordan Crouse Use the new adreno-smmu-priv fault info function to get more SMMU debug registers and print the current TTBR0 to debug per-instance pagetables and figure out which GPU block generated the request. Signed-off-by: Jordan Crouse Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 4 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 76 +++++++++++++++++++++++++-- drivers/gpu/drm/msm/msm_iommu.c | 11 +++- drivers/gpu/drm/msm/msm_mmu.h | 4 +- 4 files changed, 87 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c index ce13d49e615b..a0eef5d9b89b 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c @@ -1075,7 +1075,7 @@ bool a5xx_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring) return true; } -static int a5xx_fault_handler(void *arg, unsigned long iova, int flags) +static int a5xx_fault_handler(void *arg, unsigned long iova, int flags, void *data) { struct msm_gpu *gpu = arg; pr_warn_ratelimited("*** gpu fault: iova=%08lx, flags=%d (%u,%u,%u,%u)\n", @@ -1085,7 +1085,7 @@ static int a5xx_fault_handler(void *arg, unsigned long iova, int flags) gpu_read(gpu, REG_A5XX_CP_SCRATCH_REG(6)), gpu_read(gpu, REG_A5XX_CP_SCRATCH_REG(7))); - return -EFAULT; + return 0; } static void a5xx_cp_err_irq(struct msm_gpu *gpu) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 23464d735682..094dc17fd20f 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -959,18 +959,88 @@ static void a6xx_recover(struct msm_gpu *gpu) msm_gpu_hw_init(gpu); } -static int a6xx_fault_handler(void *arg, unsigned long iova, int flags) +static const char *a6xx_uche_fault_block(struct msm_gpu *gpu, u32 mid) +{ + static const char *uche_clients[7] = { + "VFD", "SP", "VSC", "VPC", "HLSQ", "PC", "LRZ", + }; + u32 val; + + if (mid < 1 || mid > 3) + return "UNKNOWN"; + + /* + * The source of the data depends on the mid ID read from FSYNR1. + * and the client ID read from the UCHE block + */ + val = gpu_read(gpu, REG_A6XX_UCHE_CLIENT_PF); + + /* mid = 3 is most precise and refers to only one block per client */ + if (mid == 3) + return uche_clients[val & 7]; + + /* For mid=2 the source is TP or VFD except when the client id is 0 */ + if (mid == 2) + return ((val & 7) == 0) ? "TP" : "TP|VFD"; + + /* For mid=1 just return "UCHE" as a catchall for everything else */ + return "UCHE"; +} + +static const char *a6xx_fault_block(struct msm_gpu *gpu, u32 id) +{ + if (id == 0) + return "CP"; + else if (id == 4) + return "CCU"; + else if (id == 6) + return "CDP Prefetch"; + + return a6xx_uche_fault_block(gpu, id); +} + +#define ARM_SMMU_FSR_TF BIT(1) +#define ARM_SMMU_FSR_PF BIT(3) +#define ARM_SMMU_FSR_EF BIT(4) + +static int a6xx_fault_handler(void *arg, unsigned long iova, int flags, void *data) { struct msm_gpu *gpu = arg; + struct adreno_smmu_fault_info *info = data; + const char *type = "UNKNOWN"; - pr_warn_ratelimited("*** gpu fault: iova=%08lx, flags=%d (%u,%u,%u,%u)\n", + /* + * Print a default message if we couldn't get the data from the + * adreno-smmu-priv + */ + if (!info) { + pr_warn_ratelimited("*** gpu fault: iova=%.16lx flags=%d (%u,%u,%u,%u)\n", iova, flags, gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(4)), gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(5)), gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(6)), gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(7))); - return -EFAULT; + return 0; + } + + if (info->fsr & ARM_SMMU_FSR_TF) + type = "TRANSLATION"; + else if (info->fsr & ARM_SMMU_FSR_PF) + type = "PERMISSION"; + else if (info->fsr & ARM_SMMU_FSR_EF) + type = "EXTERNAL"; + + pr_warn_ratelimited("*** gpu fault: ttbr0=%.16llx iova=%.16lx dir=%s type=%s source=%s (%u,%u,%u,%u)\n", + info->ttbr0, iova, + flags & IOMMU_FAULT_WRITE ? "WRITE" : "READ", type, + a6xx_fault_block(gpu, info->fsynr1 & 0xff), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(4)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(5)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(6)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(7))); + + return 0; } static void a6xx_cp_hw_err_irq(struct msm_gpu *gpu) diff --git a/drivers/gpu/drm/msm/msm_iommu.c b/drivers/gpu/drm/msm/msm_iommu.c index 50d881794758..6975b95c3c29 100644 --- a/drivers/gpu/drm/msm/msm_iommu.c +++ b/drivers/gpu/drm/msm/msm_iommu.c @@ -211,8 +211,17 @@ static int msm_fault_handler(struct iommu_domain *domain, struct device *dev, unsigned long iova, int flags, void *arg) { struct msm_iommu *iommu = arg; + struct adreno_smmu_priv *adreno_smmu = dev_get_drvdata(iommu->base.dev); + struct adreno_smmu_fault_info info, *ptr = NULL; + + if (adreno_smmu->get_fault_info) { + adreno_smmu->get_fault_info(adreno_smmu->cookie, &info); + ptr = &info; + } + if (iommu->base.handler) - return iommu->base.handler(iommu->base.arg, iova, flags); + return iommu->base.handler(iommu->base.arg, iova, flags, ptr); + pr_warn_ratelimited("*** fault: iova=%16lx, flags=%d\n", iova, flags); return 0; } diff --git a/drivers/gpu/drm/msm/msm_mmu.h b/drivers/gpu/drm/msm/msm_mmu.h index 61ade89d9e48..a88f44c3268d 100644 --- a/drivers/gpu/drm/msm/msm_mmu.h +++ b/drivers/gpu/drm/msm/msm_mmu.h @@ -26,7 +26,7 @@ enum msm_mmu_type { struct msm_mmu { const struct msm_mmu_funcs *funcs; struct device *dev; - int (*handler)(void *arg, unsigned long iova, int flags); + int (*handler)(void *arg, unsigned long iova, int flags, void *data); void *arg; enum msm_mmu_type type; }; @@ -43,7 +43,7 @@ struct msm_mmu *msm_iommu_new(struct device *dev, struct iommu_domain *domain); struct msm_mmu *msm_gpummu_new(struct device *dev, struct msm_gpu *gpu); static inline void msm_mmu_set_fault_handler(struct msm_mmu *mmu, void *arg, - int (*handler)(void *arg, unsigned long iova, int flags)) + int (*handler)(void *arg, unsigned long iova, int flags, void *data)) { mmu->arg = arg; mmu->handler = handler; -- 2.31.1