[{"data":1,"prerenderedAt":2931},["ShallowReactive",2],{"doc:\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis":3},{"id":4,"title":5,"body":6,"description":2919,"extension":2920,"meta":2921,"navigation":431,"path":2927,"seo":2928,"stem":2929,"__hash__":2930},"docs\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Findex.md","Batch Processing with PyQGIS",{"type":7,"value":8,"toc":2899},"minimark",[9,14,24,47,215,220,223,273,277,280,344,348,353,1160,1165,1217,1225,1229,1251,1437,1452,1690,1693,2107,2110,2152,2156,2159,2166,2176,2229,2233,2241,2303,2307,2331,2346,2350,2366,2370,2384,2391,2548,2561,2565,2568,2697,2700,2704,2736,2740,2758,2770,2790,2809,2824,2833,2837,2845,2850,2895],[10,11,13],"h1",{"id":12},"batch-processing-with-pyqgis-a-step-by-step-automation-guide","Batch Processing with PyQGIS: A Step-by-Step Automation Guide",[15,16,17,18,23],"p",{},"Automating repetitive geospatial tasks is a cornerstone of modern ",[19,20,22],"a",{"href":21},"\u002Fspatial-data-processing-automation\u002F","Spatial Data Processing & Automation with PyQGIS",". When working with dozens or hundreds of datasets, manual execution through the QGIS graphical interface quickly becomes impractical and prone to human error. Batch processing with PyQGIS bridges this gap by allowing analysts and developers to script, schedule, and scale geoprocessing operations directly within the QGIS ecosystem. This guide provides a structured workflow, tested code patterns, a process-pool approach for parallel throughput, and troubleshooting strategies to help you transition from manual clicks to reliable, reproducible spatial automation.",[15,25,26,27,31,32,36,37,41,42,46],{},"This guide sits alongside the other automation walkthroughs on this site. If you have not yet run a single algorithm programmatically, start with ",[19,28,30],{"href":29},"\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Frun-processing-algorithm-from-script\u002F","running a processing algorithm from a script",", which establishes the ",[33,34,35],"code",{},"processing.run()"," call this page repeats across a folder of inputs. When one algorithm's output must feed the next, move on to ",[19,38,40],{"href":39},"\u002Fspatial-data-processing-automation\u002Fchaining-processing-algorithms\u002F","chaining processing algorithms",". All example code is pinned to ",[43,44,45],"strong",{},"QGIS 3.34 LTR"," (Python 3.12) and noted where the API differs on the 3.28 LTR line.",[48,49,54,58,62,69,79,88,94,98,103,107,116,121,129,135,139,143,147,151,155,159,162,167,171,173,177,181,198,203,207,211],"svg",{"viewBox":50,"role":51,"ariaLabel":52,"xmlns":53},"0 0 760 360","img","A batch processing loop in PyQGIS: scan an input folder, then for each file run a processing algorithm with feedback, validate the output, and collect results into a log","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg",[55,56,57],"title",{},"PyQGIS batch processing loop",[59,60,61],"desc",{},"The diagram shows an input folder being scanned into a file list, each file flowing through a processing.run call governed by a feedback object, output validation, and a final results log.",[63,64],"rect",{"x":65,"y":65,"width":66,"height":67,"fill":68},"0","760","360","#f6f3ea",[63,70],{"x":71,"y":72,"width":73,"height":74,"rx":75,"fill":76,"stroke":77,"style":78},"24","40","150","80","8","#fffdf7","#0f766e","stroke-width:2.5",[80,81,87],"text",{"x":82,"y":83,"style":84,"fill":85,"textAnchor":86},"99","72","text-anchor:middle;font-size:14px;font-weight:bold;font-family:sans-serif","#17211d","middle","Input folder",[80,89,93],{"x":82,"y":90,"style":91,"fill":92,"textAnchor":86},"94","text-anchor:middle;font-size:12px;font-family:sans-serif","#2f3b35","glob *.gpkg",[63,95],{"x":71,"y":73,"width":73,"height":96,"rx":75,"fill":76,"stroke":97,"style":78},"70","#2563eb",[80,99,102],{"x":82,"y":100,"style":101,"fill":85,"textAnchor":86},"180","text-anchor:middle;font-size:13px;font-weight:bold;font-family:sans-serif","File list",[80,104,106],{"x":82,"y":105,"style":91,"fill":92,"textAnchor":86},"200","pathlib paths",[63,108],{"x":109,"y":110,"width":111,"height":112,"rx":113,"fill":76,"stroke":114,"style":115},"250","60","270","230","10","#b45309","stroke-width:2.5;stroke-dasharray:7 5",[80,117,120],{"x":118,"y":119,"style":101,"fill":114,"textAnchor":86},"385","86","for each file (loop)",[63,122],{"x":123,"y":124,"width":125,"height":126,"rx":75,"fill":127,"stroke":127,"style":128},"276","104","218","56","#26322d","stroke-width:2",[80,130,134],{"x":118,"y":131,"style":132,"fill":133,"textAnchor":86},"128","text-anchor:middle;font-size:13px;font-weight:bold;font-family:monospace","#d9f99d","processing.run(alg, params,",[80,136,138],{"x":118,"y":137,"style":132,"fill":133,"textAnchor":86},"146","context, feedback)",[63,140],{"x":123,"y":141,"width":125,"height":142,"rx":75,"fill":76,"stroke":77,"style":128},"176","44",[80,144,146],{"x":118,"y":145,"style":91,"fill":92,"textAnchor":86},"194","QgsProcessingFeedback:",[80,148,150],{"x":118,"y":149,"style":91,"fill":92,"textAnchor":86},"211","progress + errors",[63,152],{"x":123,"y":153,"width":125,"height":142,"rx":75,"fill":76,"stroke":154,"style":128},"234","#15803d",[80,156,158],{"x":118,"y":157,"style":91,"fill":92,"textAnchor":86},"261","validate OUTPUT exists",[63,160],{"x":161,"y":110,"width":73,"height":74,"rx":75,"fill":76,"stroke":154,"style":78},"590",[80,163,166],{"x":164,"y":165,"style":84,"fill":85,"textAnchor":86},"665","92","Output files",[80,168,170],{"x":164,"y":169,"style":91,"fill":92,"textAnchor":86},"114","written to disk",[63,172],{"x":161,"y":105,"width":73,"height":74,"rx":75,"fill":76,"stroke":97,"style":78},[80,174,176],{"x":164,"y":175,"style":84,"fill":85,"textAnchor":86},"232","Results log",[80,178,180],{"x":164,"y":179,"style":91,"fill":92,"textAnchor":86},"254","CSV \u002F JSON audit",[182,183,184],"defs",{},[185,186,194],"marker",{"id":187,"viewBox":188,"refX":189,"refY":190,"markerWidth":191,"markerHeight":191,"orient":192,"markerUnits":193},"arrow","0 0 10 10","9","5","7","auto","strokeWidth",[195,196],"path",{"d":197,"fill":92},"M0,0 L10,5 L0,10 z",[199,200],"line",{"x1":82,"y1":201,"x2":82,"y2":73,"stroke":92,"style":202},"120","stroke-width:2.5;marker-end:url(#arrow)",[199,204],{"x1":205,"y1":206,"x2":109,"y2":206,"stroke":92,"style":202},"174","185",[199,208],{"x1":209,"y1":201,"x2":161,"y2":210,"stroke":92,"style":202},"494","105",[199,212],{"x1":209,"y1":213,"x2":161,"y2":214,"stroke":92,"style":202},"256","245",[216,217,219],"h2",{"id":218},"prerequisites","Prerequisites",[15,221,222],{},"Before implementing batch workflows, ensure your environment meets the following baseline requirements:",[224,225,226,233,243,257,263],"ul",{},[227,228,229,232],"li",{},[43,230,231],{},"QGIS 3.34 LTR (or 3.28 LTR):"," PyQGIS APIs are tightly coupled with QGIS releases, so production scripts should target a Long-Term Release for API stability. The examples below are written against 3.34 LTR; on 3.28 LTR they run unchanged, and any behavioural difference is called out inline.",[227,234,235,238,239,242],{},[43,236,237],{},"Python Execution Context:"," QGIS ships with its own Python interpreter. Scripts should run inside the QGIS Python Console, the Processing Toolbox, or via standalone scripts that properly initialize the QGIS application context using ",[33,240,241],{},"QgsApplication.initQgis()",".",[227,244,245,248,249,252,253,256],{},[43,246,247],{},"Core Python Proficiency:"," Familiarity with lists, dictionaries, ",[33,250,251],{},"pathlib"," for file system operations, and ",[33,254,255],{},"try\u002Fexcept"," exception handling is essential for building resilient pipelines.",[227,258,259,262],{},[43,260,261],{},"Standardized Input Data:"," Batch operations fail predictably when file paths contain spaces, special characters, or inconsistent extensions. Organize your directory structure, use absolute paths, and validate file formats before execution.",[227,264,265,268,269,272],{},[43,266,267],{},"Processing Framework Enabled:"," The ",[33,270,271],{},"processing"," module must be imported and initialized. While modern QGIS installations include it by default, headless or custom deployments may require explicit provider registration.",[216,274,276],{"id":275},"step-by-step-workflow","Step-by-Step Workflow",[15,278,279],{},"A robust batch processing pipeline follows a consistent, repeatable sequence: environment initialization, parameter definition, iteration logic, execution, and output validation.",[281,282,283,297,306,324,333],"ol",{},[227,284,285,288,289,292,293,296],{},[43,286,287],{},"Initialize the QGIS Processing Context","\nThe Processing Framework requires a ",[33,290,291],{},"QgsProcessingContext"," and ",[33,294,295],{},"QgsProcessingFeedback"," object. The context manages layer registration, temporary file routing, and coordinate transformations, while the feedback object handles progress reporting, console logging, and cancellation signals.",[227,298,299,302,303,305],{},[43,300,301],{},"Define Input Sources and Parameters","\nUse ",[33,304,251],{}," to scan directories for target files. Filter by extension, validate file existence, and construct a structured list containing input paths and corresponding output destinations. Avoid hardcoding paths inside loops; instead, generate them dynamically based on input filenames.",[227,307,308,311,312,315,316,319,320,323],{},[43,309,310],{},"Select the Target Algorithm","\nIdentify the exact algorithm ID using the QGIS Processing Toolbox. For example, ",[33,313,314],{},"native:buffer",", ",[33,317,318],{},"gdal:cliprasterbyextent",", or ",[33,321,322],{},"native:fieldcalculator",". Always verify the ID by right-clicking the algorithm in the GUI and selecting \"Copy Algorithm ID\". Parameter keys must match the algorithm's exact specification.",[227,325,326,329,330,332],{},[43,327,328],{},"Execute in a Controlled Loop","\nIterate through your prepared dataset list. Pass parameters to ",[33,331,35],{},", capture the result dictionary, and log success or failure. Isolate each iteration so that a single malformed dataset does not halt the entire batch.",[227,334,335,338,339,343],{},[43,336,337],{},"Validate Outputs","\nAfter execution, verify that output files exist, contain expected geometry or raster bands, and align with your target ",[19,340,342],{"href":341},"\u002Fspatial-data-processing-automation\u002Fcoordinate-reference-systems\u002F","coordinate reference system",". Automated validation prevents silent data corruption and ensures downstream compatibility.",[216,345,347],{"id":346},"tested-code-pattern","Tested Code Pattern",[15,349,350,351,242],{},"The following script demonstrates a production-ready template for batch clipping vector layers. It handles context initialization, custom console feedback, error isolation, path management, and result validation. It assumes the Processing framework is already available — true inside the QGIS Python Console, but a headless run must first bootstrap it exactly as shown in ",[19,352,30],{"href":29},[354,355,360],"pre",{"className":356,"code":357,"language":358,"meta":359,"style":359},"language-python shiki shiki-themes github-dark","import pathlib\nfrom qgis.core import (\n    QgsProcessingContext,\n    QgsProcessingFeedback,\n    QgsVectorLayer,\n    QgsProject,\n)\nimport processing\n\n\nclass ConsoleFeedback(QgsProcessingFeedback):\n    \"\"\"Routes processing messages to the Python Console.\"\"\"\n\n    def setProgress(self, progress):\n        print(f\"\\rProgress: {progress:.1f}%\", end=\"\", flush=True)\n\n    def pushInfo(self, info):\n        print(f\"\\nINFO: {info}\")\n\n    def reportError(self, error, fatalError=False):\n        print(f\"\\nERROR: {error}\")\n\n\ndef batch_clip_vectors(input_dir: str, clip_layer_path: str, output_dir: str):\n    # 1. Setup context and feedback\n    context = QgsProcessingContext()\n    context.setProject(QgsProject.instance())\n    feedback = ConsoleFeedback()\n\n    # 2. Validate clip layer before iteration\n    clip_layer = QgsVectorLayer(clip_layer_path, \"clip_boundary\", \"ogr\")\n    if not clip_layer.isValid():\n        raise ValueError(f\"Invalid clip layer: {clip_layer_path}\")\n\n    # 3. Prepare paths using pathlib\n    input_path = pathlib.Path(input_dir)\n    output_path = pathlib.Path(output_dir)\n    output_path.mkdir(parents=True, exist_ok=True)\n\n    # 4. Iterate and process\n    for input_file in input_path.glob(\"*.gpkg\"):\n        out_file = output_path \u002F f\"clipped_{input_file.name}\"\n\n        params = {\n            \"INPUT\": str(input_file),\n            \"OVERLAY\": clip_layer_path,\n            \"OUTPUT\": str(out_file),\n        }\n\n        try:\n            result = processing.run(\"native:clip\", params, context=context, feedback=feedback)\n            # Validate that the algorithm actually wrote the output\n            if pathlib.Path(result.get(\"OUTPUT\", \"\")).exists():\n                print(f\"\\nSuccess: {input_file.name}\")\n            else:\n                print(f\"\\nWarning: Output missing for {input_file.name}\")\n        except Exception as e:\n            print(f\"\\nFailed: {input_file.name} | Error: {e}\")\n\n\n# Example execution (run in QGIS Python Console)\n# batch_clip_vectors(\"\u002Fdata\u002Finput_vectors\", \"\u002Fdata\u002Fclip_boundary.gpkg\", \"\u002Fdata\u002Foutput_clipped\")\n","python","",[33,361,362,374,388,394,400,406,412,418,426,433,438,456,463,468,480,540,545,556,584,589,607,634,639,644,671,678,689,695,706,711,717,738,750,777,782,788,799,810,834,839,845,865,895,900,911,925,934,947,953,958,967,1000,1006,1025,1052,1060,1086,1101,1138,1143,1148,1154],{"__ignoreMap":359},[363,364,366,370],"span",{"class":199,"line":365},1,[363,367,369],{"class":368},"snl16","import",[363,371,373],{"class":372},"s95oV"," pathlib\n",[363,375,377,380,383,385],{"class":199,"line":376},2,[363,378,379],{"class":368},"from",[363,381,382],{"class":372}," qgis.core ",[363,384,369],{"class":368},[363,386,387],{"class":372}," (\n",[363,389,391],{"class":199,"line":390},3,[363,392,393],{"class":372},"    QgsProcessingContext,\n",[363,395,397],{"class":199,"line":396},4,[363,398,399],{"class":372},"    QgsProcessingFeedback,\n",[363,401,403],{"class":199,"line":402},5,[363,404,405],{"class":372},"    QgsVectorLayer,\n",[363,407,409],{"class":199,"line":408},6,[363,410,411],{"class":372},"    QgsProject,\n",[363,413,415],{"class":199,"line":414},7,[363,416,417],{"class":372},")\n",[363,419,421,423],{"class":199,"line":420},8,[363,422,369],{"class":368},[363,424,425],{"class":372}," processing\n",[363,427,429],{"class":199,"line":428},9,[363,430,432],{"emptyLinePlaceholder":431},true,"\n",[363,434,436],{"class":199,"line":435},10,[363,437,432],{"emptyLinePlaceholder":431},[363,439,441,444,448,451,453],{"class":199,"line":440},11,[363,442,443],{"class":368},"class",[363,445,447],{"class":446},"svObZ"," ConsoleFeedback",[363,449,450],{"class":372},"(",[363,452,295],{"class":446},[363,454,455],{"class":372},"):\n",[363,457,459],{"class":199,"line":458},12,[363,460,462],{"class":461},"sU2Wk","    \"\"\"Routes processing messages to the Python Console.\"\"\"\n",[363,464,466],{"class":199,"line":465},13,[363,467,432],{"emptyLinePlaceholder":431},[363,469,471,474,477],{"class":199,"line":470},14,[363,472,473],{"class":368},"    def",[363,475,476],{"class":446}," setProgress",[363,478,479],{"class":372},"(self, progress):\n",[363,481,483,487,489,492,495,498,501,504,507,510,513,516,518,522,525,528,530,533,535,538],{"class":199,"line":482},15,[363,484,486],{"class":485},"sDLfK","        print",[363,488,450],{"class":372},[363,490,491],{"class":368},"f",[363,493,494],{"class":461},"\"",[363,496,497],{"class":485},"\\r",[363,499,500],{"class":461},"Progress: ",[363,502,503],{"class":485},"{",[363,505,506],{"class":372},"progress",[363,508,509],{"class":368},":.1f",[363,511,512],{"class":485},"}",[363,514,515],{"class":461},"%\"",[363,517,315],{"class":372},[363,519,521],{"class":520},"s9osk","end",[363,523,524],{"class":368},"=",[363,526,527],{"class":461},"\"\"",[363,529,315],{"class":372},[363,531,532],{"class":520},"flush",[363,534,524],{"class":368},[363,536,537],{"class":485},"True",[363,539,417],{"class":372},[363,541,543],{"class":199,"line":542},16,[363,544,432],{"emptyLinePlaceholder":431},[363,546,548,550,553],{"class":199,"line":547},17,[363,549,473],{"class":368},[363,551,552],{"class":446}," pushInfo",[363,554,555],{"class":372},"(self, info):\n",[363,557,559,561,563,565,567,570,573,575,578,580,582],{"class":199,"line":558},18,[363,560,486],{"class":485},[363,562,450],{"class":372},[363,564,491],{"class":368},[363,566,494],{"class":461},[363,568,569],{"class":485},"\\n",[363,571,572],{"class":461},"INFO: ",[363,574,503],{"class":485},[363,576,577],{"class":372},"info",[363,579,512],{"class":485},[363,581,494],{"class":461},[363,583,417],{"class":372},[363,585,587],{"class":199,"line":586},19,[363,588,432],{"emptyLinePlaceholder":431},[363,590,592,594,597,600,602,605],{"class":199,"line":591},20,[363,593,473],{"class":368},[363,595,596],{"class":446}," reportError",[363,598,599],{"class":372},"(self, error, fatalError",[363,601,524],{"class":368},[363,603,604],{"class":485},"False",[363,606,455],{"class":372},[363,608,610,612,614,616,618,620,623,625,628,630,632],{"class":199,"line":609},21,[363,611,486],{"class":485},[363,613,450],{"class":372},[363,615,491],{"class":368},[363,617,494],{"class":461},[363,619,569],{"class":485},[363,621,622],{"class":461},"ERROR: ",[363,624,503],{"class":485},[363,626,627],{"class":372},"error",[363,629,512],{"class":485},[363,631,494],{"class":461},[363,633,417],{"class":372},[363,635,637],{"class":199,"line":636},22,[363,638,432],{"emptyLinePlaceholder":431},[363,640,642],{"class":199,"line":641},23,[363,643,432],{"emptyLinePlaceholder":431},[363,645,647,650,653,656,659,662,664,667,669],{"class":199,"line":646},24,[363,648,649],{"class":368},"def",[363,651,652],{"class":446}," batch_clip_vectors",[363,654,655],{"class":372},"(input_dir: ",[363,657,658],{"class":485},"str",[363,660,661],{"class":372},", clip_layer_path: ",[363,663,658],{"class":485},[363,665,666],{"class":372},", output_dir: ",[363,668,658],{"class":485},[363,670,455],{"class":372},[363,672,674],{"class":199,"line":673},25,[363,675,677],{"class":676},"sjoCn","    # 1. Setup context and feedback\n",[363,679,681,684,686],{"class":199,"line":680},26,[363,682,683],{"class":372},"    context ",[363,685,524],{"class":368},[363,687,688],{"class":372}," QgsProcessingContext()\n",[363,690,692],{"class":199,"line":691},27,[363,693,694],{"class":372},"    context.setProject(QgsProject.instance())\n",[363,696,698,701,703],{"class":199,"line":697},28,[363,699,700],{"class":372},"    feedback ",[363,702,524],{"class":368},[363,704,705],{"class":372}," ConsoleFeedback()\n",[363,707,709],{"class":199,"line":708},29,[363,710,432],{"emptyLinePlaceholder":431},[363,712,714],{"class":199,"line":713},30,[363,715,716],{"class":676},"    # 2. Validate clip layer before iteration\n",[363,718,720,723,725,728,731,733,736],{"class":199,"line":719},31,[363,721,722],{"class":372},"    clip_layer ",[363,724,524],{"class":368},[363,726,727],{"class":372}," QgsVectorLayer(clip_layer_path, ",[363,729,730],{"class":461},"\"clip_boundary\"",[363,732,315],{"class":372},[363,734,735],{"class":461},"\"ogr\"",[363,737,417],{"class":372},[363,739,741,744,747],{"class":199,"line":740},32,[363,742,743],{"class":368},"    if",[363,745,746],{"class":368}," not",[363,748,749],{"class":372}," clip_layer.isValid():\n",[363,751,753,756,759,761,763,766,768,771,773,775],{"class":199,"line":752},33,[363,754,755],{"class":368},"        raise",[363,757,758],{"class":485}," ValueError",[363,760,450],{"class":372},[363,762,491],{"class":368},[363,764,765],{"class":461},"\"Invalid clip layer: ",[363,767,503],{"class":485},[363,769,770],{"class":372},"clip_layer_path",[363,772,512],{"class":485},[363,774,494],{"class":461},[363,776,417],{"class":372},[363,778,780],{"class":199,"line":779},34,[363,781,432],{"emptyLinePlaceholder":431},[363,783,785],{"class":199,"line":784},35,[363,786,787],{"class":676},"    # 3. Prepare paths using pathlib\n",[363,789,791,794,796],{"class":199,"line":790},36,[363,792,793],{"class":372},"    input_path ",[363,795,524],{"class":368},[363,797,798],{"class":372}," pathlib.Path(input_dir)\n",[363,800,802,805,807],{"class":199,"line":801},37,[363,803,804],{"class":372},"    output_path ",[363,806,524],{"class":368},[363,808,809],{"class":372}," pathlib.Path(output_dir)\n",[363,811,813,816,819,821,823,825,828,830,832],{"class":199,"line":812},38,[363,814,815],{"class":372},"    output_path.mkdir(",[363,817,818],{"class":520},"parents",[363,820,524],{"class":368},[363,822,537],{"class":485},[363,824,315],{"class":372},[363,826,827],{"class":520},"exist_ok",[363,829,524],{"class":368},[363,831,537],{"class":485},[363,833,417],{"class":372},[363,835,837],{"class":199,"line":836},39,[363,838,432],{"emptyLinePlaceholder":431},[363,840,842],{"class":199,"line":841},40,[363,843,844],{"class":676},"    # 4. Iterate and process\n",[363,846,848,851,854,857,860,863],{"class":199,"line":847},41,[363,849,850],{"class":368},"    for",[363,852,853],{"class":372}," input_file ",[363,855,856],{"class":368},"in",[363,858,859],{"class":372}," input_path.glob(",[363,861,862],{"class":461},"\"*.gpkg\"",[363,864,455],{"class":372},[363,866,868,871,873,876,879,882,885,887,890,892],{"class":199,"line":867},42,[363,869,870],{"class":372},"        out_file ",[363,872,524],{"class":368},[363,874,875],{"class":372}," output_path ",[363,877,878],{"class":368},"\u002F",[363,880,881],{"class":368}," f",[363,883,884],{"class":461},"\"clipped_",[363,886,503],{"class":485},[363,888,889],{"class":372},"input_file.name",[363,891,512],{"class":485},[363,893,894],{"class":461},"\"\n",[363,896,898],{"class":199,"line":897},43,[363,899,432],{"emptyLinePlaceholder":431},[363,901,903,906,908],{"class":199,"line":902},44,[363,904,905],{"class":372},"        params ",[363,907,524],{"class":368},[363,909,910],{"class":372}," {\n",[363,912,914,917,920,922],{"class":199,"line":913},45,[363,915,916],{"class":461},"            \"INPUT\"",[363,918,919],{"class":372},": ",[363,921,658],{"class":485},[363,923,924],{"class":372},"(input_file),\n",[363,926,928,931],{"class":199,"line":927},46,[363,929,930],{"class":461},"            \"OVERLAY\"",[363,932,933],{"class":372},": clip_layer_path,\n",[363,935,937,940,942,944],{"class":199,"line":936},47,[363,938,939],{"class":461},"            \"OUTPUT\"",[363,941,919],{"class":372},[363,943,658],{"class":485},[363,945,946],{"class":372},"(out_file),\n",[363,948,950],{"class":199,"line":949},48,[363,951,952],{"class":372},"        }\n",[363,954,956],{"class":199,"line":955},49,[363,957,432],{"emptyLinePlaceholder":431},[363,959,961,964],{"class":199,"line":960},50,[363,962,963],{"class":368},"        try",[363,965,966],{"class":372},":\n",[363,968,970,973,975,978,981,984,987,989,992,995,997],{"class":199,"line":969},51,[363,971,972],{"class":372},"            result ",[363,974,524],{"class":368},[363,976,977],{"class":372}," processing.run(",[363,979,980],{"class":461},"\"native:clip\"",[363,982,983],{"class":372},", params, ",[363,985,986],{"class":520},"context",[363,988,524],{"class":368},[363,990,991],{"class":372},"context, ",[363,993,994],{"class":520},"feedback",[363,996,524],{"class":368},[363,998,999],{"class":372},"feedback)\n",[363,1001,1003],{"class":199,"line":1002},52,[363,1004,1005],{"class":676},"            # Validate that the algorithm actually wrote the output\n",[363,1007,1009,1012,1015,1018,1020,1022],{"class":199,"line":1008},53,[363,1010,1011],{"class":368},"            if",[363,1013,1014],{"class":372}," pathlib.Path(result.get(",[363,1016,1017],{"class":461},"\"OUTPUT\"",[363,1019,315],{"class":372},[363,1021,527],{"class":461},[363,1023,1024],{"class":372},")).exists():\n",[363,1026,1028,1031,1033,1035,1037,1039,1042,1044,1046,1048,1050],{"class":199,"line":1027},54,[363,1029,1030],{"class":485},"                print",[363,1032,450],{"class":372},[363,1034,491],{"class":368},[363,1036,494],{"class":461},[363,1038,569],{"class":485},[363,1040,1041],{"class":461},"Success: ",[363,1043,503],{"class":485},[363,1045,889],{"class":372},[363,1047,512],{"class":485},[363,1049,494],{"class":461},[363,1051,417],{"class":372},[363,1053,1055,1058],{"class":199,"line":1054},55,[363,1056,1057],{"class":368},"            else",[363,1059,966],{"class":372},[363,1061,1063,1065,1067,1069,1071,1073,1076,1078,1080,1082,1084],{"class":199,"line":1062},56,[363,1064,1030],{"class":485},[363,1066,450],{"class":372},[363,1068,491],{"class":368},[363,1070,494],{"class":461},[363,1072,569],{"class":485},[363,1074,1075],{"class":461},"Warning: Output missing for ",[363,1077,503],{"class":485},[363,1079,889],{"class":372},[363,1081,512],{"class":485},[363,1083,494],{"class":461},[363,1085,417],{"class":372},[363,1087,1089,1092,1095,1098],{"class":199,"line":1088},57,[363,1090,1091],{"class":368},"        except",[363,1093,1094],{"class":485}," Exception",[363,1096,1097],{"class":368}," as",[363,1099,1100],{"class":372}," e:\n",[363,1102,1104,1107,1109,1111,1113,1115,1118,1120,1122,1124,1127,1129,1132,1134,1136],{"class":199,"line":1103},58,[363,1105,1106],{"class":485},"            print",[363,1108,450],{"class":372},[363,1110,491],{"class":368},[363,1112,494],{"class":461},[363,1114,569],{"class":485},[363,1116,1117],{"class":461},"Failed: ",[363,1119,503],{"class":485},[363,1121,889],{"class":372},[363,1123,512],{"class":485},[363,1125,1126],{"class":461}," | Error: ",[363,1128,503],{"class":485},[363,1130,1131],{"class":372},"e",[363,1133,512],{"class":485},[363,1135,494],{"class":461},[363,1137,417],{"class":372},[363,1139,1141],{"class":199,"line":1140},59,[363,1142,432],{"emptyLinePlaceholder":431},[363,1144,1146],{"class":199,"line":1145},60,[363,1147,432],{"emptyLinePlaceholder":431},[363,1149,1151],{"class":199,"line":1150},61,[363,1152,1153],{"class":676},"# Example execution (run in QGIS Python Console)\n",[363,1155,1157],{"class":199,"line":1156},62,[363,1158,1159],{"class":676},"# batch_clip_vectors(\"\u002Fdata\u002Finput_vectors\", \"\u002Fdata\u002Fclip_boundary.gpkg\", \"\u002Fdata\u002Foutput_clipped\")\n",[1161,1162,1164],"h3",{"id":1163},"code-breakdown","Code Breakdown",[224,1166,1167,1180,1192,1205],{},[227,1168,1169,1172,1173,1175,1176,1179],{},[43,1170,1171],{},"Context & Feedback Objects:"," ",[33,1174,291],{}," acts as the execution environment, tracking temporary layers, managing memory, and handling CRS transformations. The custom ",[33,1177,1178],{},"ConsoleFeedback"," class ensures progress and errors print directly to the console instead of being swallowed by the default silent implementation.",[227,1181,1182,1172,1185,1187,1188,1191],{},[43,1183,1184],{},"Path Handling:",[33,1186,251],{}," replaces legacy ",[33,1189,1190],{},"os.path"," operations, offering safer path resolution, automatic directory creation, and cleaner string interpolation.",[227,1193,1194,1172,1197,1199,1200,292,1202,1204],{},[43,1195,1196],{},"Algorithm Execution:",[33,1198,35],{}," is the core dispatcher. It accepts a dictionary of parameters matching the algorithm's specification. Passing ",[33,1201,986],{},[33,1203,994],{}," ensures the script integrates cleanly with QGIS's internal state management rather than operating in isolation.",[227,1206,1207,1210,1211,1213,1214,1216],{},[43,1208,1209],{},"Error Isolation:"," Wrapping ",[33,1212,35],{}," in a ",[33,1215,255],{}," block prevents a single invalid dataset from halting the entire batch. This pattern is critical when scaling to hundreds of files.",[15,1218,1219,1220,1224],{},"When adapting this template for ",[19,1221,1223],{"href":1222},"\u002Fspatial-data-processing-automation\u002Fvector-data-manipulation\u002F","vector data manipulation",", simply swap the algorithm ID and adjust the parameter keys. The same structural pattern applies to topology checks, attribute joins, and geometry validation routines.",[216,1226,1228],{"id":1227},"parallelizing-the-batch","Parallelizing the Batch",[15,1230,1231,1232,1234,1235,1238,1239,1242,1243,1245,1246,1250],{},"The sequential loop above processes one file at a time. For CPU-bound algorithms — dissolve, buffer on dense geometries, complex raster math — that leaves most of your cores idle. The instinct is to reach for threads, but this is a trap in PyQGIS: the Processing objects (",[33,1233,291],{},", the algorithm instances, and much of the underlying Qt\u002FGDAL state) are ",[43,1236,1237],{},"not thread-safe",", and a single ",[33,1240,1241],{},"QgsApplication"," cannot be shared across threads that all call ",[33,1244,35],{},". The reliable pattern is ",[1247,1248,1249],"em",{},"process-level"," parallelism, where each worker is its own operating-system process with its own headless QGIS.",[48,1252,1255,1258,1261,1268,1272,1277,1281,1285,1290,1294,1301,1304,1308,1313,1318,1323,1327,1332,1334,1338,1341,1345,1348,1352,1356,1358,1361,1366,1370,1373,1375,1377,1381,1385,1389,1392,1396,1399,1402,1405,1409,1412,1416,1420,1423,1425,1429,1433],{"viewBox":1253,"role":51,"ariaLabel":1254,"xmlns":53},"0 0 800 462","Sequential batch versus a process pool: the same set of input files is either processed one at a time by a single QGIS process, or split across four headless QGIS worker processes running in parallel, with both paths writing to one shared results log",[55,1256,1257],{},"Sequential loop versus process-pool parallelism in PyQGIS",[59,1259,1260],{},"A comparison diagram. The same queue of input files feeds two execution models. On the left, one QGIS process runs a serial loop, clipping files one after another so wall time is the sum of every file's time and other CPU cores sit idle. On the right, a process pool starts four headless QGIS workers that each clip a subset simultaneously, so wall time is roughly the slowest single file. Both paths converge on one shared run_log.csv.",[182,1262,1263],{},[185,1264,1266],{"id":1265,"viewBox":188,"refX":189,"refY":190,"markerWidth":191,"markerHeight":191,"orient":192,"markerUnits":193},"bp-arrow",[195,1267],{"d":197,"fill":92},[63,1269],{"x":65,"y":65,"width":1270,"height":1271,"fill":68},"800","462",[63,1273],{"x":1274,"y":1275,"width":105,"height":1276,"rx":75,"fill":76,"stroke":97,"style":78},"300","18","46",[80,1278,1280],{"x":1279,"y":72,"style":101,"fill":85,"textAnchor":86},"400","N input files",[80,1282,1284],{"x":1279,"y":1283,"style":91,"fill":92,"textAnchor":86},"57","glob *.gpkg jobs",[199,1286],{"x1":67,"y1":110,"x2":1287,"y2":1288,"stroke":92,"style":1289},"215","100","stroke-width:2.5;marker-end:url(#bp-arrow)",[199,1291],{"x1":1292,"y1":110,"x2":1293,"y2":1288,"stroke":92,"style":1289},"440","585",[63,1295],{"x":1296,"y":1288,"width":1297,"height":1298,"rx":113,"fill":1299,"stroke":114,"style":1300},"30","350","262","none","stroke-width:1.5;stroke-dasharray:6 5",[63,1302],{"x":1296,"y":1288,"width":1297,"height":1303,"rx":113,"fill":114},"32",[63,1305],{"x":1296,"y":1306,"width":1297,"height":1307,"fill":114},"116","16",[80,1309,1312],{"x":1310,"y":1311,"style":101,"fill":76,"textAnchor":86},"205","121","Sequential loop — one process",[63,1314],{"x":1315,"y":137,"width":1316,"height":1296,"rx":1317,"fill":127},"48","314","6",[80,1319,1322],{"x":1310,"y":1320,"style":1321,"fill":133,"textAnchor":86},"166","text-anchor:middle;font-size:12px;font-family:monospace","1× initQgis()  →  serial for-loop",[63,1324],{"x":1315,"y":1325,"width":1326,"height":72,"rx":1317,"fill":76,"stroke":114,"style":128},"192","66",[80,1328,1331],{"x":1329,"y":1330,"style":91,"fill":85,"textAnchor":86},"81","217","clip f1",[63,1333],{"x":131,"y":1325,"width":1326,"height":72,"rx":1317,"fill":76,"stroke":114,"style":128},[80,1335,1337],{"x":1336,"y":1330,"style":91,"fill":85,"textAnchor":86},"161","clip f2",[63,1339],{"x":1340,"y":1325,"width":1326,"height":72,"rx":1317,"fill":76,"stroke":114,"style":128},"208",[80,1342,1344],{"x":1343,"y":1330,"style":91,"fill":85,"textAnchor":86},"241","clip f3",[63,1346],{"x":1347,"y":1325,"width":1326,"height":72,"rx":1317,"fill":76,"stroke":114,"style":128},"288",[80,1349,1351],{"x":1350,"y":1330,"style":91,"fill":85,"textAnchor":86},"321","clip f4",[199,1353],{"x1":169,"y1":1354,"x2":131,"y2":1354,"stroke":92,"style":1355},"212","stroke-width:2;marker-end:url(#bp-arrow)",[199,1357],{"x1":145,"y1":1354,"x2":1340,"y2":1354,"stroke":92,"style":1355},[199,1359],{"x1":1360,"y1":1354,"x2":1347,"y2":1354,"stroke":92,"style":1355},"274",[80,1362,1365],{"x":1310,"y":1363,"style":1364,"fill":114,"textAnchor":86},"266","text-anchor:middle;font-size:12px;font-weight:bold;font-family:sans-serif","wall time = t1 + t2 + t3 + t4",[80,1367,1369],{"x":1310,"y":1368,"style":91,"fill":92,"textAnchor":86},"290","other CPU cores sit idle",[63,1371],{"x":1372,"y":1288,"width":1297,"height":1298,"rx":113,"fill":1299,"stroke":154,"style":1300},"420",[63,1374],{"x":1372,"y":1288,"width":1297,"height":1303,"rx":113,"fill":154},[63,1376],{"x":1372,"y":1306,"width":1297,"height":1307,"fill":154},[80,1378,1380],{"x":1379,"y":1311,"style":101,"fill":76,"textAnchor":86},"595","Process pool — 4 workers",[63,1382],{"x":1383,"y":137,"width":1316,"height":1384,"rx":1317,"fill":127},"438","28",[80,1386,1388],{"x":1379,"y":1387,"style":1321,"fill":133,"textAnchor":86},"165","queue: (in, overlay, out) tuples",[63,1390],{"x":1383,"y":1391,"width":1316,"height":1296,"rx":1317,"fill":76,"stroke":154,"style":128},"186",[80,1393,1395],{"x":1379,"y":1310,"style":1394,"fill":85,"textAnchor":86},"text-anchor:middle;font-size:11.5px;font-family:sans-serif","worker 1 · own QGIS · clip f1",[63,1397],{"x":1383,"y":1398,"width":1316,"height":1296,"rx":1317,"fill":76,"stroke":154,"style":128},"222",[80,1400,1401],{"x":1379,"y":1343,"style":1394,"fill":85,"textAnchor":86},"worker 2 · own QGIS · clip f2",[63,1403],{"x":1383,"y":1404,"width":1316,"height":1296,"rx":1317,"fill":76,"stroke":154,"style":128},"258",[80,1406,1408],{"x":1379,"y":1407,"style":1394,"fill":85,"textAnchor":86},"277","worker 3 · own QGIS · clip f3",[63,1410],{"x":1383,"y":1411,"width":1316,"height":1296,"rx":1317,"fill":76,"stroke":154,"style":128},"294",[80,1413,1415],{"x":1379,"y":1414,"style":1394,"fill":85,"textAnchor":86},"313","worker 4 · own QGIS · clip f4",[80,1417,1419],{"x":1379,"y":1418,"style":1364,"fill":154,"textAnchor":86},"346","wall time ≈ max(t1 … t4)",[199,1421],{"x1":1310,"y1":1422,"x2":67,"y2":1279,"stroke":92,"style":1289},"362",[199,1424],{"x1":1379,"y1":1422,"x2":1292,"y2":1279,"stroke":92,"style":1289},[63,1426],{"x":1368,"y":1427,"width":1428,"height":1276,"rx":75,"fill":76,"stroke":97,"style":78},"404","220",[80,1430,1432],{"x":1279,"y":1431,"style":101,"fill":85,"textAnchor":86},"426","run_log.csv",[80,1434,1436],{"x":1279,"y":1435,"style":91,"fill":92,"textAnchor":86},"443","one row per file",[15,1438,1439,1440,1443,1444,1447,1448,1451],{},"The cleanest way to express this is ",[33,1441,1442],{},"concurrent.futures.ProcessPoolExecutor"," with an ",[33,1445,1446],{},"initializer"," that stands up QGIS ",[43,1449,1450],{},"once per worker process",". Put the worker code in its own importable module so the pool can pickle a reference to the function:",[354,1453,1455],{"className":356,"code":1454,"language":358,"meta":359,"style":359},"# clip_worker.py — one QGIS bootstrap per pool process\nfrom qgis.core import QgsApplication\nfrom processing.core.Processing import Processing\nimport processing\n\n_qgs = None\n\n\ndef init_worker():\n    \"\"\"Runs once in each pool process: stand up a headless QGIS.\"\"\"\n    global _qgs\n    _qgs = QgsApplication([], False)\n    _qgs.initQgis()\n    Processing.initialize()\n\n\ndef clip_one(job):\n    \"\"\"Process a single file. `job` is a plain tuple so it pickles\n    cleanly across the process boundary — never pass live QGIS objects.\"\"\"\n    input_file, overlay, output_file = job\n    try:\n        processing.run(\n            \"native:clip\",\n            {\"INPUT\": input_file, \"OVERLAY\": overlay, \"OUTPUT\": output_file},\n        )\n        return (input_file, \"ok\", output_file)\n    except Exception as e:  # isolate: one bad file must not kill the pool\n        return (input_file, \"error\", str(e))\n",[33,1456,1457,1462,1473,1485,1491,1495,1505,1509,1513,1523,1528,1536,1550,1555,1560,1564,1568,1578,1583,1588,1598,1605,1610,1618,1640,1645,1659,1674],{"__ignoreMap":359},[363,1458,1459],{"class":199,"line":365},[363,1460,1461],{"class":676},"# clip_worker.py — one QGIS bootstrap per pool process\n",[363,1463,1464,1466,1468,1470],{"class":199,"line":376},[363,1465,379],{"class":368},[363,1467,382],{"class":372},[363,1469,369],{"class":368},[363,1471,1472],{"class":372}," QgsApplication\n",[363,1474,1475,1477,1480,1482],{"class":199,"line":390},[363,1476,379],{"class":368},[363,1478,1479],{"class":372}," processing.core.Processing ",[363,1481,369],{"class":368},[363,1483,1484],{"class":372}," Processing\n",[363,1486,1487,1489],{"class":199,"line":396},[363,1488,369],{"class":368},[363,1490,425],{"class":372},[363,1492,1493],{"class":199,"line":402},[363,1494,432],{"emptyLinePlaceholder":431},[363,1496,1497,1500,1502],{"class":199,"line":408},[363,1498,1499],{"class":372},"_qgs ",[363,1501,524],{"class":368},[363,1503,1504],{"class":485}," None\n",[363,1506,1507],{"class":199,"line":414},[363,1508,432],{"emptyLinePlaceholder":431},[363,1510,1511],{"class":199,"line":420},[363,1512,432],{"emptyLinePlaceholder":431},[363,1514,1515,1517,1520],{"class":199,"line":428},[363,1516,649],{"class":368},[363,1518,1519],{"class":446}," init_worker",[363,1521,1522],{"class":372},"():\n",[363,1524,1525],{"class":199,"line":435},[363,1526,1527],{"class":461},"    \"\"\"Runs once in each pool process: stand up a headless QGIS.\"\"\"\n",[363,1529,1530,1533],{"class":199,"line":440},[363,1531,1532],{"class":368},"    global",[363,1534,1535],{"class":372}," _qgs\n",[363,1537,1538,1541,1543,1546,1548],{"class":199,"line":458},[363,1539,1540],{"class":372},"    _qgs ",[363,1542,524],{"class":368},[363,1544,1545],{"class":372}," QgsApplication([], ",[363,1547,604],{"class":485},[363,1549,417],{"class":372},[363,1551,1552],{"class":199,"line":465},[363,1553,1554],{"class":372},"    _qgs.initQgis()\n",[363,1556,1557],{"class":199,"line":470},[363,1558,1559],{"class":372},"    Processing.initialize()\n",[363,1561,1562],{"class":199,"line":482},[363,1563,432],{"emptyLinePlaceholder":431},[363,1565,1566],{"class":199,"line":542},[363,1567,432],{"emptyLinePlaceholder":431},[363,1569,1570,1572,1575],{"class":199,"line":547},[363,1571,649],{"class":368},[363,1573,1574],{"class":446}," clip_one",[363,1576,1577],{"class":372},"(job):\n",[363,1579,1580],{"class":199,"line":558},[363,1581,1582],{"class":461},"    \"\"\"Process a single file. `job` is a plain tuple so it pickles\n",[363,1584,1585],{"class":199,"line":586},[363,1586,1587],{"class":461},"    cleanly across the process boundary — never pass live QGIS objects.\"\"\"\n",[363,1589,1590,1593,1595],{"class":199,"line":591},[363,1591,1592],{"class":372},"    input_file, overlay, output_file ",[363,1594,524],{"class":368},[363,1596,1597],{"class":372}," job\n",[363,1599,1600,1603],{"class":199,"line":609},[363,1601,1602],{"class":368},"    try",[363,1604,966],{"class":372},[363,1606,1607],{"class":199,"line":636},[363,1608,1609],{"class":372},"        processing.run(\n",[363,1611,1612,1615],{"class":199,"line":641},[363,1613,1614],{"class":461},"            \"native:clip\"",[363,1616,1617],{"class":372},",\n",[363,1619,1620,1623,1626,1629,1632,1635,1637],{"class":199,"line":646},[363,1621,1622],{"class":372},"            {",[363,1624,1625],{"class":461},"\"INPUT\"",[363,1627,1628],{"class":372},": input_file, ",[363,1630,1631],{"class":461},"\"OVERLAY\"",[363,1633,1634],{"class":372},": overlay, ",[363,1636,1017],{"class":461},[363,1638,1639],{"class":372},": output_file},\n",[363,1641,1642],{"class":199,"line":673},[363,1643,1644],{"class":372},"        )\n",[363,1646,1647,1650,1653,1656],{"class":199,"line":680},[363,1648,1649],{"class":368},"        return",[363,1651,1652],{"class":372}," (input_file, ",[363,1654,1655],{"class":461},"\"ok\"",[363,1657,1658],{"class":372},", output_file)\n",[363,1660,1661,1664,1666,1668,1671],{"class":199,"line":691},[363,1662,1663],{"class":368},"    except",[363,1665,1094],{"class":485},[363,1667,1097],{"class":368},[363,1669,1670],{"class":372}," e:  ",[363,1672,1673],{"class":676},"# isolate: one bad file must not kill the pool\n",[363,1675,1676,1678,1680,1683,1685,1687],{"class":199,"line":697},[363,1677,1649],{"class":368},[363,1679,1652],{"class":372},[363,1681,1682],{"class":461},"\"error\"",[363,1684,315],{"class":372},[363,1686,658],{"class":485},[363,1688,1689],{"class":372},"(e))\n",[15,1691,1692],{},"The driver builds a list of plain-tuple jobs, fans them out across the pool, and collects one result row per file:",[354,1694,1696],{"className":356,"code":1695,"language":358,"meta":359,"style":359},"import csv\nimport pathlib\nfrom concurrent.futures import ProcessPoolExecutor\n\nfrom clip_worker import init_worker, clip_one\n\n\ndef run_parallel(input_dir, overlay, output_dir, workers=4):\n    input_dir = pathlib.Path(input_dir)\n    output_dir = pathlib.Path(output_dir)\n    output_dir.mkdir(parents=True, exist_ok=True)\n\n    jobs = [\n        (str(f), overlay, str(output_dir \u002F f\"clipped_{f.name}\"))\n        for f in input_dir.glob(\"*.gpkg\")\n    ]\n\n    results = []\n    with ProcessPoolExecutor(max_workers=workers, initializer=init_worker) as pool:\n        for row in pool.map(clip_one, jobs):\n            print(f\"{row[1].upper():7} {pathlib.Path(row[0]).name}\")\n            results.append(row)\n\n    with open(output_dir \u002F \"run_log.csv\", \"w\", newline=\"\") as fh:\n        writer = csv.writer(fh)\n        writer.writerow([\"input\", \"status\", \"detail\"])\n        writer.writerows(results)\n    return results\n\n\nif __name__ == \"__main__\":  # required on Windows\u002FmacOS (spawn start method)\n    run_parallel(\"\u002Fdata\u002Finput_vectors\", \"\u002Fdata\u002Fclip_boundary.gpkg\", \"\u002Fdata\u002Fout\")\n",[33,1697,1698,1705,1711,1723,1727,1739,1743,1747,1764,1773,1782,1803,1807,1817,1850,1867,1872,1876,1886,1915,1927,1970,1975,1979,2015,2025,2046,2051,2059,2063,2067,2087],{"__ignoreMap":359},[363,1699,1700,1702],{"class":199,"line":365},[363,1701,369],{"class":368},[363,1703,1704],{"class":372}," csv\n",[363,1706,1707,1709],{"class":199,"line":376},[363,1708,369],{"class":368},[363,1710,373],{"class":372},[363,1712,1713,1715,1718,1720],{"class":199,"line":390},[363,1714,379],{"class":368},[363,1716,1717],{"class":372}," concurrent.futures ",[363,1719,369],{"class":368},[363,1721,1722],{"class":372}," ProcessPoolExecutor\n",[363,1724,1725],{"class":199,"line":396},[363,1726,432],{"emptyLinePlaceholder":431},[363,1728,1729,1731,1734,1736],{"class":199,"line":402},[363,1730,379],{"class":368},[363,1732,1733],{"class":372}," clip_worker ",[363,1735,369],{"class":368},[363,1737,1738],{"class":372}," init_worker, clip_one\n",[363,1740,1741],{"class":199,"line":408},[363,1742,432],{"emptyLinePlaceholder":431},[363,1744,1745],{"class":199,"line":414},[363,1746,432],{"emptyLinePlaceholder":431},[363,1748,1749,1751,1754,1757,1759,1762],{"class":199,"line":420},[363,1750,649],{"class":368},[363,1752,1753],{"class":446}," run_parallel",[363,1755,1756],{"class":372},"(input_dir, overlay, output_dir, workers",[363,1758,524],{"class":368},[363,1760,1761],{"class":485},"4",[363,1763,455],{"class":372},[363,1765,1766,1769,1771],{"class":199,"line":428},[363,1767,1768],{"class":372},"    input_dir ",[363,1770,524],{"class":368},[363,1772,798],{"class":372},[363,1774,1775,1778,1780],{"class":199,"line":435},[363,1776,1777],{"class":372},"    output_dir ",[363,1779,524],{"class":368},[363,1781,809],{"class":372},[363,1783,1784,1787,1789,1791,1793,1795,1797,1799,1801],{"class":199,"line":440},[363,1785,1786],{"class":372},"    output_dir.mkdir(",[363,1788,818],{"class":520},[363,1790,524],{"class":368},[363,1792,537],{"class":485},[363,1794,315],{"class":372},[363,1796,827],{"class":520},[363,1798,524],{"class":368},[363,1800,537],{"class":485},[363,1802,417],{"class":372},[363,1804,1805],{"class":199,"line":458},[363,1806,432],{"emptyLinePlaceholder":431},[363,1808,1809,1812,1814],{"class":199,"line":465},[363,1810,1811],{"class":372},"    jobs ",[363,1813,524],{"class":368},[363,1815,1816],{"class":372}," [\n",[363,1818,1819,1822,1824,1827,1829,1832,1834,1836,1838,1840,1843,1845,1847],{"class":199,"line":470},[363,1820,1821],{"class":372},"        (",[363,1823,658],{"class":485},[363,1825,1826],{"class":372},"(f), overlay, ",[363,1828,658],{"class":485},[363,1830,1831],{"class":372},"(output_dir ",[363,1833,878],{"class":368},[363,1835,881],{"class":368},[363,1837,884],{"class":461},[363,1839,503],{"class":485},[363,1841,1842],{"class":372},"f.name",[363,1844,512],{"class":485},[363,1846,494],{"class":461},[363,1848,1849],{"class":372},"))\n",[363,1851,1852,1855,1858,1860,1863,1865],{"class":199,"line":482},[363,1853,1854],{"class":368},"        for",[363,1856,1857],{"class":372}," f ",[363,1859,856],{"class":368},[363,1861,1862],{"class":372}," input_dir.glob(",[363,1864,862],{"class":461},[363,1866,417],{"class":372},[363,1868,1869],{"class":199,"line":542},[363,1870,1871],{"class":372},"    ]\n",[363,1873,1874],{"class":199,"line":547},[363,1875,432],{"emptyLinePlaceholder":431},[363,1877,1878,1881,1883],{"class":199,"line":558},[363,1879,1880],{"class":372},"    results ",[363,1882,524],{"class":368},[363,1884,1885],{"class":372}," []\n",[363,1887,1888,1891,1894,1897,1899,1902,1904,1906,1909,1912],{"class":199,"line":586},[363,1889,1890],{"class":368},"    with",[363,1892,1893],{"class":372}," ProcessPoolExecutor(",[363,1895,1896],{"class":520},"max_workers",[363,1898,524],{"class":368},[363,1900,1901],{"class":372},"workers, ",[363,1903,1446],{"class":520},[363,1905,524],{"class":368},[363,1907,1908],{"class":372},"init_worker) ",[363,1910,1911],{"class":368},"as",[363,1913,1914],{"class":372}," pool:\n",[363,1916,1917,1919,1922,1924],{"class":199,"line":591},[363,1918,1854],{"class":368},[363,1920,1921],{"class":372}," row ",[363,1923,856],{"class":368},[363,1925,1926],{"class":372}," pool.map(clip_one, jobs):\n",[363,1928,1929,1931,1933,1935,1937,1939,1942,1945,1948,1951,1953,1956,1959,1961,1964,1966,1968],{"class":199,"line":609},[363,1930,1106],{"class":485},[363,1932,450],{"class":372},[363,1934,491],{"class":368},[363,1936,494],{"class":461},[363,1938,503],{"class":485},[363,1940,1941],{"class":372},"row[",[363,1943,1944],{"class":485},"1",[363,1946,1947],{"class":372},"].upper()",[363,1949,1950],{"class":368},":7",[363,1952,512],{"class":485},[363,1954,1955],{"class":485}," {",[363,1957,1958],{"class":372},"pathlib.Path(row[",[363,1960,65],{"class":485},[363,1962,1963],{"class":372},"]).name",[363,1965,512],{"class":485},[363,1967,494],{"class":461},[363,1969,417],{"class":372},[363,1971,1972],{"class":199,"line":636},[363,1973,1974],{"class":372},"            results.append(row)\n",[363,1976,1977],{"class":199,"line":641},[363,1978,432],{"emptyLinePlaceholder":431},[363,1980,1981,1983,1986,1988,1990,1993,1995,1998,2000,2003,2005,2007,2010,2012],{"class":199,"line":646},[363,1982,1890],{"class":368},[363,1984,1985],{"class":485}," open",[363,1987,1831],{"class":372},[363,1989,878],{"class":368},[363,1991,1992],{"class":461}," \"run_log.csv\"",[363,1994,315],{"class":372},[363,1996,1997],{"class":461},"\"w\"",[363,1999,315],{"class":372},[363,2001,2002],{"class":520},"newline",[363,2004,524],{"class":368},[363,2006,527],{"class":461},[363,2008,2009],{"class":372},") ",[363,2011,1911],{"class":368},[363,2013,2014],{"class":372}," fh:\n",[363,2016,2017,2020,2022],{"class":199,"line":673},[363,2018,2019],{"class":372},"        writer ",[363,2021,524],{"class":368},[363,2023,2024],{"class":372}," csv.writer(fh)\n",[363,2026,2027,2030,2033,2035,2038,2040,2043],{"class":199,"line":680},[363,2028,2029],{"class":372},"        writer.writerow([",[363,2031,2032],{"class":461},"\"input\"",[363,2034,315],{"class":372},[363,2036,2037],{"class":461},"\"status\"",[363,2039,315],{"class":372},[363,2041,2042],{"class":461},"\"detail\"",[363,2044,2045],{"class":372},"])\n",[363,2047,2048],{"class":199,"line":691},[363,2049,2050],{"class":372},"        writer.writerows(results)\n",[363,2052,2053,2056],{"class":199,"line":697},[363,2054,2055],{"class":368},"    return",[363,2057,2058],{"class":372}," results\n",[363,2060,2061],{"class":199,"line":708},[363,2062,432],{"emptyLinePlaceholder":431},[363,2064,2065],{"class":199,"line":713},[363,2066,432],{"emptyLinePlaceholder":431},[363,2068,2069,2072,2075,2078,2081,2084],{"class":199,"line":719},[363,2070,2071],{"class":368},"if",[363,2073,2074],{"class":485}," __name__",[363,2076,2077],{"class":368}," ==",[363,2079,2080],{"class":461}," \"__main__\"",[363,2082,2083],{"class":372},":  ",[363,2085,2086],{"class":676},"# required on Windows\u002FmacOS (spawn start method)\n",[363,2088,2089,2092,2095,2097,2100,2102,2105],{"class":199,"line":740},[363,2090,2091],{"class":372},"    run_parallel(",[363,2093,2094],{"class":461},"\"\u002Fdata\u002Finput_vectors\"",[363,2096,315],{"class":372},[363,2098,2099],{"class":461},"\"\u002Fdata\u002Fclip_boundary.gpkg\"",[363,2101,315],{"class":372},[363,2103,2104],{"class":461},"\"\u002Fdata\u002Fout\"",[363,2106,417],{"class":372},[15,2108,2109],{},"Four practical rules keep a parallel batch honest:",[224,2111,2112,2122,2132,2138],{},[227,2113,2114,2117,2118,2121],{},[43,2115,2116],{},"Never send live QGIS objects between processes."," Pass strings and numbers; each worker opens its own layers. Contexts, feedback objects, and ",[33,2119,2120],{},"QgsVectorLayer"," instances cannot be pickled and must be created inside the worker.",[227,2123,2124,2127,2128,2131],{},[43,2125,2126],{},"Don't oversubscribe."," GDAL and some native algorithms already spawn their own threads, so setting ",[33,2129,2130],{},"workers"," equal to your core count can slow the batch down through contention. Start at half your physical cores and measure.",[227,2133,2134,2137],{},[43,2135,2136],{},"Parallelism helps CPU-bound work, not I\u002FO-bound work."," If the bottleneck is reading and writing large files from a single spinning disk or a network share, more processes just queue at the storage layer — batch to a fast local SSD first (see the memory section below).",[227,2139,2140,2143,2144,2147,2148,2151],{},[43,2141,2142],{},"Guard the entry point."," On Windows and macOS the pool uses the ",[33,2145,2146],{},"spawn"," start method, which re-imports the driver module in each child; the ",[33,2149,2150],{},"if __name__ == \"__main__\""," guard prevents an infinite spawn loop.",[216,2153,2155],{"id":2154},"common-errors-and-fixes","Common Errors and Fixes",[15,2157,2158],{},"Batch processing with PyQGIS introduces specific failure modes that rarely appear during manual GUI operations. Understanding these patterns saves hours of debugging and ensures pipeline reliability.",[1161,2160,2162,2163],{"id":2161},"_1-qgsprocessingexception-algorithm-not-found","1. ",[33,2164,2165],{},"QgsProcessingException: Algorithm not found",[15,2167,2168,2171,2172,2175],{},[43,2169,2170],{},"Cause:"," The algorithm ID is misspelled, or the required provider (e.g., GDAL, GRASS, SAGA) is not loaded in the current session.\n",[43,2173,2174],{},"Fix:"," Verify the ID in the Processing Toolbox. For third-party providers, ensure the plugin is enabled in Settings > Plugins. You can programmatically check availability before execution:",[354,2177,2179],{"className":356,"code":2178,"language":358,"meta":359,"style":359},"from qgis.core import QgsApplication\nalg = QgsApplication.processingRegistry().algorithmById(\"native:clip\")\nif not alg:\n    raise RuntimeError(\"Target algorithm is not registered in this environment\")\n",[33,2180,2181,2191,2205,2214],{"__ignoreMap":359},[363,2182,2183,2185,2187,2189],{"class":199,"line":365},[363,2184,379],{"class":368},[363,2186,382],{"class":372},[363,2188,369],{"class":368},[363,2190,1472],{"class":372},[363,2192,2193,2196,2198,2201,2203],{"class":199,"line":376},[363,2194,2195],{"class":372},"alg ",[363,2197,524],{"class":368},[363,2199,2200],{"class":372}," QgsApplication.processingRegistry().algorithmById(",[363,2202,980],{"class":461},[363,2204,417],{"class":372},[363,2206,2207,2209,2211],{"class":199,"line":390},[363,2208,2071],{"class":368},[363,2210,746],{"class":368},[363,2212,2213],{"class":372}," alg:\n",[363,2215,2216,2219,2222,2224,2227],{"class":199,"line":396},[363,2217,2218],{"class":368},"    raise",[363,2220,2221],{"class":485}," RuntimeError",[363,2223,450],{"class":372},[363,2225,2226],{"class":461},"\"Target algorithm is not registered in this environment\"",[363,2228,417],{"class":372},[1161,2230,2232],{"id":2231},"_2-silent-failures-with-empty-outputs","2. Silent Failures with Empty Outputs",[15,2234,2235,2237,2238,2240],{},[43,2236,2170],{}," The input geometry is invalid, a spatial reference mismatch prevents intersection, or the output directory lacks write permissions.\n",[43,2239,2174],{}," Always validate geometry and feature counts before processing. Additionally, explicitly set the target CRS in the processing context if your workflow requires on-the-fly transformation:",[354,2242,2244],{"className":356,"code":2243,"language":358,"meta":359,"style":359},"from qgis.core import QgsCoordinateReferenceSystem\nif not layer.isValid() or layer.featureCount() == 0:\n    raise ValueError(\"Layer is invalid or empty\")\ncontext.setDestinationCrs(QgsCoordinateReferenceSystem(\"EPSG:32633\"))\n",[33,2245,2246,2257,2280,2293],{"__ignoreMap":359},[363,2247,2248,2250,2252,2254],{"class":199,"line":365},[363,2249,379],{"class":368},[363,2251,382],{"class":372},[363,2253,369],{"class":368},[363,2255,2256],{"class":372}," QgsCoordinateReferenceSystem\n",[363,2258,2259,2261,2263,2266,2269,2272,2275,2278],{"class":199,"line":376},[363,2260,2071],{"class":368},[363,2262,746],{"class":368},[363,2264,2265],{"class":372}," layer.isValid() ",[363,2267,2268],{"class":368},"or",[363,2270,2271],{"class":372}," layer.featureCount() ",[363,2273,2274],{"class":368},"==",[363,2276,2277],{"class":485}," 0",[363,2279,966],{"class":372},[363,2281,2282,2284,2286,2288,2291],{"class":199,"line":390},[363,2283,2218],{"class":368},[363,2285,758],{"class":485},[363,2287,450],{"class":372},[363,2289,2290],{"class":461},"\"Layer is invalid or empty\"",[363,2292,417],{"class":372},[363,2294,2295,2298,2301],{"class":199,"line":396},[363,2296,2297],{"class":372},"context.setDestinationCrs(QgsCoordinateReferenceSystem(",[363,2299,2300],{"class":461},"\"EPSG:32633\"",[363,2302,1849],{"class":372},[1161,2304,2306],{"id":2305},"_3-memory-exhaustion-on-large-datasets","3. Memory Exhaustion on Large Datasets",[15,2308,2309,2311,2312,2315,2316,2319,2320,2322,2323,2325,2326,2330],{},[43,2310,2170],{}," QGIS loads layers into memory by default. Processing hundreds of large GeoTIFFs or highly complex polygons can trigger ",[33,2313,2314],{},"std::bad_alloc"," or Python ",[33,2317,2318],{},"MemoryError",".\n",[43,2321,2174],{}," Use the ",[33,2324,291],{}," temporary output management and enable disk-based processing where possible. For raster-heavy tasks, consider chunking your inputs or leveraging ",[19,2327,2329],{"href":2328},"\u002Fspatial-data-processing-automation\u002Fraster-analysis-workflows\u002F","Raster Analysis Workflows"," that utilize GDAL's virtual raster (VRT) tiling to avoid loading entire datasets into RAM. You can also direct temporary outputs to a fast disk:",[354,2332,2334],{"className":356,"code":2333,"language":358,"meta":359,"style":359},"context.setTemporaryDirectory(\"\u002Fpath\u002Fto\u002Ffast_ssd\u002Ftemp_qgis\")\n",[33,2335,2336],{"__ignoreMap":359},[363,2337,2338,2341,2344],{"class":199,"line":365},[363,2339,2340],{"class":372},"context.setTemporaryDirectory(",[363,2342,2343],{"class":461},"\"\u002Fpath\u002Fto\u002Ffast_ssd\u002Ftemp_qgis\"",[363,2345,417],{"class":372},[1161,2347,2349],{"id":2348},"_4-progress-feedback-not-updating","4. Progress Feedback Not Updating",[15,2351,2352,268,2354,2356,2357,2359,2360,2362,2363,2365],{},[43,2353,2170],{},[33,2355,295],{}," object is instantiated but not passed to ",[33,2358,35],{},", or the script runs outside the main QGIS thread without proper signal routing.\n",[43,2361,2174],{}," Always pass the ",[33,2364,994],{}," argument explicitly. If running in a standalone script or custom GUI, implement a custom feedback class that logs to a file or console (as demonstrated in the code pattern above).",[216,2367,2369],{"id":2368},"scaling-and-integration-patterns","Scaling and Integration Patterns",[15,2371,2372,2373,2376,2377,2380,2381,2383],{},"Once your batch scripts are stable, integrate them into broader automation pipelines. For enterprise deployments, wrap your PyQGIS batch functions in a CLI tool using ",[33,2374,2375],{},"argparse"," or ",[33,2378,2379],{},"click",". Schedule execution via cron, systemd timers, or CI\u002FCD pipelines. Where a single file must pass through several algorithms in sequence, promote the per-file body of your loop into a proper pipeline as described in ",[19,2382,40],{"href":39}," — batch iteration and algorithm chaining compose cleanly, with the chain running once inside each loop pass.",[15,2385,2386,2387,2390],{},"Always log outputs to a structured format (JSON or CSV) rather than bare ",[33,2388,2389],{},"print()"," calls, so you can track success rates, processing times, and error frequencies across runs. A minimal, auditable record captures the input, the outcome, the output path, and how long each item took:",[354,2392,2394],{"className":356,"code":2393,"language":358,"meta":359,"style":359},"import csv\nimport time\n\ndef log_result(writer, input_file, status, output, started):\n    writer.writerow([input_file, status, output, round(time.perf_counter() - started, 2)])\n\nwith open(\"batch_log.csv\", \"w\", newline=\"\") as fh:\n    writer = csv.writer(fh)\n    writer.writerow([\"input\", \"status\", \"output\", \"seconds\"])\n    for input_file in files:\n        started = time.perf_counter()\n        # ... processing.run(...) inside try\u002Fexcept ...\n        log_result(writer, input_file, \"ok\", out_file, started)\n",[33,2395,2396,2402,2409,2413,2423,2446,2450,2480,2489,2512,2523,2533,2538],{"__ignoreMap":359},[363,2397,2398,2400],{"class":199,"line":365},[363,2399,369],{"class":368},[363,2401,1704],{"class":372},[363,2403,2404,2406],{"class":199,"line":376},[363,2405,369],{"class":368},[363,2407,2408],{"class":372}," time\n",[363,2410,2411],{"class":199,"line":390},[363,2412,432],{"emptyLinePlaceholder":431},[363,2414,2415,2417,2420],{"class":199,"line":396},[363,2416,649],{"class":368},[363,2418,2419],{"class":446}," log_result",[363,2421,2422],{"class":372},"(writer, input_file, status, output, started):\n",[363,2424,2425,2428,2431,2434,2437,2440,2443],{"class":199,"line":402},[363,2426,2427],{"class":372},"    writer.writerow([input_file, status, output, ",[363,2429,2430],{"class":485},"round",[363,2432,2433],{"class":372},"(time.perf_counter() ",[363,2435,2436],{"class":368},"-",[363,2438,2439],{"class":372}," started, ",[363,2441,2442],{"class":485},"2",[363,2444,2445],{"class":372},")])\n",[363,2447,2448],{"class":199,"line":408},[363,2449,432],{"emptyLinePlaceholder":431},[363,2451,2452,2455,2457,2459,2462,2464,2466,2468,2470,2472,2474,2476,2478],{"class":199,"line":414},[363,2453,2454],{"class":368},"with",[363,2456,1985],{"class":485},[363,2458,450],{"class":372},[363,2460,2461],{"class":461},"\"batch_log.csv\"",[363,2463,315],{"class":372},[363,2465,1997],{"class":461},[363,2467,315],{"class":372},[363,2469,2002],{"class":520},[363,2471,524],{"class":368},[363,2473,527],{"class":461},[363,2475,2009],{"class":372},[363,2477,1911],{"class":368},[363,2479,2014],{"class":372},[363,2481,2482,2485,2487],{"class":199,"line":420},[363,2483,2484],{"class":372},"    writer ",[363,2486,524],{"class":368},[363,2488,2024],{"class":372},[363,2490,2491,2494,2496,2498,2500,2502,2505,2507,2510],{"class":199,"line":428},[363,2492,2493],{"class":372},"    writer.writerow([",[363,2495,2032],{"class":461},[363,2497,315],{"class":372},[363,2499,2037],{"class":461},[363,2501,315],{"class":372},[363,2503,2504],{"class":461},"\"output\"",[363,2506,315],{"class":372},[363,2508,2509],{"class":461},"\"seconds\"",[363,2511,2045],{"class":372},[363,2513,2514,2516,2518,2520],{"class":199,"line":435},[363,2515,850],{"class":368},[363,2517,853],{"class":372},[363,2519,856],{"class":368},[363,2521,2522],{"class":372}," files:\n",[363,2524,2525,2528,2530],{"class":199,"line":440},[363,2526,2527],{"class":372},"        started ",[363,2529,524],{"class":368},[363,2531,2532],{"class":372}," time.perf_counter()\n",[363,2534,2535],{"class":199,"line":458},[363,2536,2537],{"class":676},"        # ... processing.run(...) inside try\u002Fexcept ...\n",[363,2539,2540,2543,2545],{"class":199,"line":465},[363,2541,2542],{"class":372},"        log_result(writer, input_file, ",[363,2544,1655],{"class":461},[363,2546,2547],{"class":372},", out_file, started)\n",[15,2549,2550,2551,2555,2556,2560],{},"When designing automated cartographic outputs, remember that batch processing extends beyond data transformation. You can script map generation by iterating through filtered datasets, applying dynamic styles, and exporting PDFs or images without manual intervention — the same loop structure drives ",[19,2552,2554],{"href":2553},"\u002Fspatial-data-processing-automation\u002Fautomated-map-layout-generation\u002F","automated map layout generation",", and for one layout repeated over many features it graduates naturally into ",[19,2557,2559],{"href":2558},"\u002Fspatial-data-processing-automation\u002Fautomating-atlas-map-series\u002F","automating an atlas map series",". This capability transforms static mapping workflows into dynamic, data-driven publishing pipelines.",[216,2562,2564],{"id":2563},"designing-a-batch-that-can-be-re-run","Designing a batch that can be re-run",[15,2566,2567],{},"A batch that must start from scratch after every failure is unusable on any real dataset. Three properties turn it into something you can run repeatedly without thinking.",[15,2569,2570],{},[48,2571,2574,2577,2580,2583,2592,2597,2602,2607,2638,2642,2646,2648,2651,2667,2670,2673,2677,2680,2684,2686,2690,2694],{"viewBox":2572,"role":51,"ariaLabel":2573,"xmlns":53},"0 0 760 244","Three properties of a re-runnable batch: skipping outputs that already exist, recording failures instead of aborting, and writing to a temporary name before renaming",[55,2575,2576],{},"Three properties of a re-runnable batch",[59,2578,2579],{},"Idempotence means checking whether the output already exists and skipping that input. Fault isolation means recording a failure and continuing rather than aborting the run. Atomic writes mean writing to a temporary filename and renaming on success, so a half-written file is never mistaken for a finished one.",[63,2581],{"x":65,"y":65,"width":66,"height":2582,"fill":68},"244",[182,2584,2585],{},[185,2586,2589],{"id":2587,"viewBox":188,"refX":75,"refY":190,"markerWidth":191,"markerHeight":191,"orient":2588},"batchRerunArrow","auto-start-reverse",[195,2590],{"d":2591,"fill":77},"M0 0 L10 5 L0 10 z",[80,2593,2596],{"x":2594,"y":2595,"style":84,"fill":85,"textAnchor":86},"380","26","Run it twice and nothing should break",[63,2598],{"x":2599,"y":1276,"width":2600,"height":2601,"rx":113,"fill":76,"stroke":154,"style":78},"12","236","182",[80,2603,2606],{"x":2604,"y":96,"style":2605,"fill":154,"textAnchor":86},"130","text-anchor:middle;font-size:11.5px;font-weight:bold;font-family:sans-serif","idempotent",[2608,2609,2610,2616,2621,2624,2626,2630,2634,2636],"g",{},[63,2611],{"x":72,"y":2612,"width":74,"height":1296,"rx":190,"fill":2613,"fillOpacity":2614,"stroke":2613,"style":2615},"88","#59645f",0.2,"stroke-width:1.4",[80,2617,2620],{"x":74,"y":2618,"style":2619,"fill":92,"textAnchor":86},"108","text-anchor:middle;font-size:10px;font-family:sans-serif","done",[63,2622],{"x":2623,"y":2612,"width":74,"height":1296,"rx":190,"fill":2613,"fillOpacity":2614,"stroke":2613,"style":2615},"140",[80,2625,2620],{"x":100,"y":2618,"style":2619,"fill":92,"textAnchor":86},[63,2627],{"x":72,"y":131,"width":74,"height":1296,"rx":190,"fill":154,"fillOpacity":2628,"stroke":154,"style":2629},0.4,"stroke-width:1.6",[80,2631,2633],{"x":74,"y":2632,"style":2619,"fill":92,"textAnchor":86},"148","todo",[63,2635],{"x":2623,"y":131,"width":74,"height":1296,"rx":190,"fill":154,"fillOpacity":2628,"stroke":154,"style":2629},[80,2637,2633],{"x":100,"y":2632,"style":2619,"fill":92,"textAnchor":86},[80,2639,2641],{"x":2604,"y":1391,"style":2640,"fill":92,"textAnchor":86},"text-anchor:middle;font-size:10.5px;font-family:sans-serif","skip what already exists",[80,2643,2645],{"x":2604,"y":2644,"style":2640,"fill":154,"textAnchor":86},"206","a re-run resumes",[63,2647],{"x":1298,"y":1276,"width":2600,"height":2601,"rx":113,"fill":76,"stroke":97,"style":78},[80,2649,2650],{"x":2594,"y":96,"style":2605,"fill":97,"textAnchor":86},"fault isolated",[2608,2652,2653,2657,2661,2664],{},[63,2654],{"x":1368,"y":2655,"width":100,"height":2595,"rx":190,"fill":154,"fillOpacity":2656,"stroke":154,"style":2615},"96",0.35,[63,2658],{"x":1368,"y":131,"width":100,"height":2595,"rx":190,"fill":2659,"fillOpacity":2660,"stroke":2659,"style":2629},"#b91c1c",0.3,[80,2662,2663],{"x":2594,"y":137,"style":2619,"fill":92,"textAnchor":86},"recorded, not fatal",[63,2665],{"x":1368,"y":2666,"width":100,"height":2595,"rx":190,"fill":154,"fillOpacity":2656,"stroke":154,"style":2615},"160",[80,2668,2669],{"x":2594,"y":2644,"style":2640,"fill":97,"textAnchor":86},"one bad input is not the run",[63,2671],{"x":2672,"y":1276,"width":2600,"height":2601,"rx":113,"fill":76,"stroke":114,"style":78},"512",[80,2674,2676],{"x":2675,"y":96,"style":2605,"fill":114,"textAnchor":86},"630","atomic",[63,2678],{"x":2679,"y":2655,"width":100,"height":1296,"rx":190,"fill":114,"fillOpacity":2614,"stroke":114,"style":2629},"540",[80,2681,2683],{"x":2675,"y":1306,"style":2682,"fill":92,"textAnchor":86},"text-anchor:middle;font-size:10px;font-family:monospace","out.gpkg.tmp",[63,2685],{"x":2679,"y":73,"width":100,"height":1296,"rx":190,"fill":154,"fillOpacity":2660,"stroke":154,"style":2629},[80,2687,2689],{"x":2675,"y":2688,"style":2682,"fill":92,"textAnchor":86},"170","out.gpkg",[199,2691],{"x1":2675,"y1":2692,"x2":2675,"y2":137,"stroke":77,"style":2693},"126","stroke-width:2;marker-end:url(#batchRerunArrow)",[80,2695,2696],{"x":2675,"y":2644,"style":2640,"fill":114,"textAnchor":86},"rename only on success",[15,2698,2699],{},"The third is the one most often skipped and the one that causes the worst failures: without it, a run interrupted halfway leaves a truncated output that the next run's existence check happily accepts as finished.",[216,2701,2703],{"id":2702},"key-takeaways","Key Takeaways",[224,2705,2706,2709,2720,2726,2733],{},[227,2707,2708],{},"Batch processing with PyQGIS turns repetitive, click-driven geoprocessing into reliable, reproducible, and auditable workflows that scale from a handful of files to enterprise datasets.",[227,2710,2711,2712,292,2714,2716,2717,2719],{},"Build every batch on the Processing framework's ",[33,2713,291],{},[33,2715,295],{}," objects, and pass them explicitly to ",[33,2718,35],{}," so state management and progress reporting work as intended.",[227,2721,2722,2723,2725],{},"Isolate each iteration in a ",[33,2724,255],{}," block: a single malformed dataset should be logged and skipped, never allowed to halt the whole run.",[227,2727,2728,2729,2732],{},"Parallelize with a ",[1247,2730,2731],{},"process"," pool, not threads — give each worker its own headless QGIS, pass only picklable data across the boundary, and don't oversubscribe cores that GDAL is already using.",[227,2734,2735],{},"Validate outputs at every stage (existence, feature or band counts, and the target CRS), and write a structured CSV or JSON log so large runs stay auditable.",[216,2737,2739],{"id":2738},"frequently-asked-questions","Frequently Asked Questions",[15,2741,2742,2748,2749,2376,2752,2754,2755,242],{},[43,2743,2744,2745,2747],{},"How do I find the exact algorithm ID to pass to ",[33,2746,35],{},"?","\nOpen the Processing Toolbox in QGIS, right-click the algorithm you want, and choose \"Copy Algorithm ID\". The ID includes the provider prefix, such as ",[33,2750,2751],{},"native:clip",[33,2753,318],{},". You can also enumerate all registered algorithms programmatically with ",[33,2756,2757],{},"QgsApplication.processingRegistry().algorithms()",[15,2759,2760,2763,2764,2766,2767,2769],{},[43,2761,2762],{},"Why does my batch run continue silently after a single file fails?","\nThat is the intended design of the tested pattern: each ",[33,2765,35],{}," call is wrapped in a ",[33,2768,255],{}," block so one malformed dataset cannot halt the entire batch. The failure is logged per file rather than raised globally. Remove or narrow the exception handling only if you want the batch to stop on the first error.",[15,2771,2772,2775,2776,2779,2780,2783,2784,2787,2788,242],{},[43,2773,2774],{},"Do I need to run batch scripts inside QGIS, or can they run headless?","\nBoth work. Inside the QGIS Python Console the application context already exists, but for cron jobs, CI\u002FCD, or Docker you must bootstrap it yourself with ",[33,2777,2778],{},"QgsApplication([], False)"," followed by ",[33,2781,2782],{},"initQgis()",", and call ",[33,2785,2786],{},"exitQgis()"," when finished. You also need to import and initialize the Processing framework before calling ",[33,2789,35],{},[15,2791,2792,2795,2796,2798,2799,2801,2802,2804,2805,2808],{},[43,2793,2794],{},"Can I run batch jobs in parallel to use all my CPU cores?","\nYes, but use ",[1247,2797,2731],{}," parallelism, not threads. QGIS Processing objects are not thread-safe and a single ",[33,2800,1241],{}," cannot be shared across threads calling ",[33,2803,35],{},". Use ",[33,2806,2807],{},"ProcessPoolExecutor"," with an initializer that stands up a headless QGIS once per worker process, pass only picklable data (strings, numbers) to each job, and start with roughly half your physical cores because GDAL already threads internally. See the \"Parallelizing the Batch\" section for the full pattern.",[15,2810,2811,2819,2820,2823],{},[43,2812,2813,2814,2376,2816,2818],{},"What causes ",[33,2815,2314],{},[33,2817,2318],{}," on large batches?","\nQGIS loads layers into memory by default, so processing many large rasters or complex polygons can exhaust available RAM. Direct temporary outputs to a fast disk with ",[33,2821,2822],{},"context.setTemporaryDirectory()",", write outputs to files rather than memory layers, and chunk or tile raster inputs. For raster-heavy work, VRT tiling avoids loading entire datasets at once.",[15,2825,2826,2829,2830,2832],{},[43,2827,2828],{},"How should I log results so I can audit large runs?","\nReplace ad-hoc ",[33,2831,2389],{}," calls with structured logging to CSV or JSON, capturing the input path, success or failure status, output path, and elapsed time per item. This produces an auditable trail of success rates and error frequencies that scales to thousands of files and makes troubleshooting failed items straightforward.",[216,2834,2836],{"id":2835},"related-guides","Related Guides",[15,2838,2839,1172,2842,2844],{},[43,2840,2841],{},"Up:",[19,2843,22],{"href":21}," — the overview guide this page belongs to.",[15,2846,2847],{},[43,2848,2849],{},"Go deeper:",[224,2851,2852,2858,2864,2870,2876,2882,2888],{},[227,2853,2854,2857],{},[19,2855,2856],{"href":29},"Run a Processing Algorithm from a Script in PyQGIS"," — the single-run bootstrap this batch repeats.",[227,2859,2860,2863],{},[19,2861,2862],{"href":39},"Chaining Processing Algorithms in PyQGIS"," — compose several algorithms into one pipeline inside each loop pass.",[227,2865,2866,2869],{},[19,2867,2868],{"href":1222},"Vector Data Manipulation in PyQGIS"," — swap in the vector algorithms your batch drives.",[227,2871,2872,2875],{},[19,2873,2874],{"href":2328},"Raster Analysis Workflows in PyQGIS"," — memory-safe patterns for batching large rasters.",[227,2877,2878,2881],{},[19,2879,2880],{"href":341},"Coordinate Reference Systems in PyQGIS"," — reproject and validate CRS across a batch.",[227,2883,2884,2887],{},[19,2885,2886],{"href":2553},"Automated Map Layout Generation"," — apply the same loop to exporting maps.",[227,2889,2890,2894],{},[19,2891,2893],{"href":2892},"\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Frun-algorithm-over-folder-of-files-pyqgis\u002F","Run an Algorithm Over a Folder of Files in PyQGIS"," — discover inputs, survive a bad file, and report what happened.",[2896,2897,2898],"style",{},"html pre.shiki code .snl16, html code.shiki .snl16{--shiki-default:#F97583}html pre.shiki code .s95oV, html code.shiki .s95oV{--shiki-default:#E1E4E8}html pre.shiki code .svObZ, html code.shiki .svObZ{--shiki-default:#B392F0}html pre.shiki code .sU2Wk, html code.shiki .sU2Wk{--shiki-default:#9ECBFF}html pre.shiki code .sDLfK, html code.shiki .sDLfK{--shiki-default:#79B8FF}html pre.shiki code .s9osk, html code.shiki .s9osk{--shiki-default:#FFAB70}html pre.shiki code .sjoCn, html code.shiki .sjoCn{--shiki-default:#9AA79F}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":359,"searchDepth":376,"depth":376,"links":2900},[2901,2902,2903,2906,2907,2914,2915,2916,2917,2918],{"id":218,"depth":376,"text":219},{"id":275,"depth":376,"text":276},{"id":346,"depth":376,"text":347,"children":2904},[2905],{"id":1163,"depth":390,"text":1164},{"id":1227,"depth":376,"text":1228},{"id":2154,"depth":376,"text":2155,"children":2908},[2909,2911,2912,2913],{"id":2161,"depth":390,"text":2910},"1. QgsProcessingException: Algorithm not found",{"id":2231,"depth":390,"text":2232},{"id":2305,"depth":390,"text":2306},{"id":2348,"depth":390,"text":2349},{"id":2368,"depth":376,"text":2369},{"id":2563,"depth":376,"text":2564},{"id":2702,"depth":376,"text":2703},{"id":2738,"depth":376,"text":2739},{"id":2835,"depth":376,"text":2836},"Step-by-step guide to batch processing in PyQGIS. Run processing algorithms across many datasets, parallelize tasks with a process pool, isolate errors, and automate GIS workflows.","md",{"slug":2922,"type":2923,"breadcrumb":2924,"datePublished":2925,"dateModified":2926},"batch-processing-with-pyqgis","guide","Batch Processing","2025-03-11","2026-07-18","\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis",{"title":5,"description":2919},"spatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Findex","-kpX-GxwN4i5XOZ0TMJCeEdDzfdMO6nt2c32MriJtCQ",1787823363461]