From owner-tech+M40246@openbsd.org Mon Dec 29 05:57:02 2014 Delivered-To: david@gwynne.id.au Received: by 10.152.48.80 with SMTP id j16csp2656401lan; Sun, 28 Dec 2014 11:57:03 -0800 (PST) X-Received: by 10.42.236.75 with SMTP id kj11mr40031737icb.47.1419796622825; Sun, 28 Dec 2014 11:57:02 -0800 (PST) Return-Path: Received: from shear.ucar.edu (lists.openbsd.org. [192.43.244.163]) by mx.google.com with ESMTPS id u39si23680735ioi.98.2014.12.28.11.57.02 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 28 Dec 2014 11:57:02 -0800 (PST) Received-SPF: pass (google.com: manual fallback record for domain of owner-tech+M40246@openbsd.org designates 192.43.244.163 as permitted sender) client-ip=192.43.244.163; Authentication-Results: mx.google.com; spf=pass (google.com: manual fallback record for domain of owner-tech+M40246@openbsd.org designates 192.43.244.163 as permitted sender) smtp.mail=owner-tech+M40246@openbsd.org Received: from openbsd.org (localhost [127.0.0.1]) by shear.ucar.edu (8.14.8/8.14.5) with ESMTP id sBSKA7o5015812; Sun, 28 Dec 2014 13:10:07 -0700 (MST) Received: from glazunov.sibelius.xs4all.nl (sibelius.xs4all.nl [83.163.83.176]) by shear.ucar.edu (8.14.8/8.14.8) with ESMTP id sBSK9wOU004824 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO) for ; Sun, 28 Dec 2014 13:10:00 -0700 (MST) Received: from glazunov.sibelius.xs4all.nl (kettenis@localhost [127.0.0.1]) by glazunov.sibelius.xs4all.nl (8.14.5/8.14.3) with ESMTP id sBSJuYkG008409 for ; Sun, 28 Dec 2014 20:56:34 +0100 (CET) Received: (from kettenis@localhost) by glazunov.sibelius.xs4all.nl (8.14.5/8.14.3/Submit) id sBSJuXkX014608; Sun, 28 Dec 2014 20:56:33 +0100 (CET) Date: Sun, 28 Dec 2014 20:56:33 +0100 (CET) Message-Id: <201412281956.sBSJuXkX014608@glazunov.sibelius.xs4all.nl> From: Mark Kettenis To: tech@openbsd.org Subject: km_alloc/km_free optimization List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Loop: tech@openbsd.org Precedence: list Sender: owner-tech@openbsd.org Status: RO Content-Length: 3368 Lines: 124 One side-effect of dlg@'s "* 8" pool diff is that some pools now use "large" pool pages that aren't directly mapped on PMAP_DIRECT architecture. In general it's not possible to map "large" pages directly since the individual memory pages are not guaranteed to be contigious. However, the mbuf cluster pools use the kp_dma_contig constraint, which does guarantee that the pages are contigious. The diff below makes us use direct mappings for those pool pages. This should make allocating/deallocating mbuf clusters faster and decrease kva pressure on PMAP_DIRECT architecture. In the process I deleted the disabled "dubious optimization" code. Nobody picked this up in the last couple of years since art@ committed this code. ok? P.S. I would not be surprised if this would "fix" the problems landisk has with the "* 8" pool diff. Currently updating my landisk to verify. Index: uvm_km.c =================================================================== RCS file: /home/cvs/src/sys/uvm/uvm_km.c,v retrieving revision 1.123 diff -u -p -r1.123 uvm_km.c --- uvm_km.c 17 Dec 2014 06:58:11 -0000 1.123 +++ uvm_km.c 28 Dec 2014 19:37:42 -0000 @@ -811,9 +811,6 @@ km_alloc(size_t sz, const struct kmem_va vm_prot_t prot; int pla_flags; int pla_maxseg; -#ifdef __HAVE_PMAP_DIRECT - paddr_t pa; -#endif vaddr_t va, sva; KASSERT(sz == round_page(sz)); @@ -841,33 +838,18 @@ km_alloc(size_t sz, const struct kmem_va #ifdef __HAVE_PMAP_DIRECT if (kv->kv_align) goto alloc_va; -#if 1 - /* - * For now, only do DIRECT mappings for single page - * allocations, until we figure out a good way to deal - * with contig allocations in km_free. - */ - if (!kv->kv_singlepage) - goto alloc_va; -#endif + /* - * Dubious optimization. If we got a contig segment, just map it - * through the direct map. + * Only use direct mappings for single page or single segment + * allocations. */ - TAILQ_FOREACH(pg, &pgl, pageq) { - if (pg != TAILQ_FIRST(&pgl) && - VM_PAGE_TO_PHYS(pg) != pa + PAGE_SIZE) - break; - pa = VM_PAGE_TO_PHYS(pg); - } - if (pg == NULL) { + if (kv->kv_singlepage || kp->kp_maxseg == 1) { TAILQ_FOREACH(pg, &pgl, pageq) { - vaddr_t v; - v = pmap_map_direct(pg); + va = pmap_map_direct(pg); if (pg == TAILQ_FIRST(&pgl)) - va = v; + sva = va; } - return ((void *)va); + return ((void *)sva); } #endif alloc_va: @@ -947,28 +929,35 @@ km_free(void *v, size_t sz, const struct struct vm_page *pg; struct pglist pgl; - sva = va = (vaddr_t)v; - eva = va + sz; + sva = (vaddr_t)v; + eva = sva + sz; - if (kp->kp_nomem) { + if (kp->kp_nomem) goto free_va; - } - if (kv->kv_singlepage) { #ifdef __HAVE_PMAP_DIRECT - pg = pmap_unmap_direct(va); - uvm_pagefree(pg); + if (kv->kv_singlepage || kp->kp_maxseg == 1) { + TAILQ_INIT(&pgl); + for (va = sva; va < eva; va += PAGE_SIZE) { + pg = pmap_unmap_direct(va); + TAILQ_INSERT_TAIL(&pgl, pg, pageq); + } + uvm_pglistfree(&pgl); + return; + } #else + if (kv->kv_singlepage) { struct uvm_km_free_page *fp = v; + mtx_enter(&uvm_km_pages.mtx); fp->next = uvm_km_pages.freelist; uvm_km_pages.freelist = fp; if (uvm_km_pages.freelistlen++ > 16) wakeup(&uvm_km_pages.km_proc); mtx_leave(&uvm_km_pages.mtx); -#endif return; } +#endif if (kp->kp_pageable) { pmap_remove(pmap_kernel(), sva, eva);