{"id":12048,"date":"2026-08-19T14:20:57","date_gmt":"2026-08-19T14:20:57","guid":{"rendered":"https:\/\/cyber-valley.de\/?post_type=researchfund-project&#038;p=12048"},"modified":"2026-09-01T12:10:16","modified_gmt":"2026-09-01T12:10:16","slug":"x_resource-efficient-autonomous-training-algorithms-for-deep-learning","status":"publish","type":"researchfund-project","link":"https:\/\/cyber-valley.de\/de\/researchfund-project\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\/","title":{"rendered":"Resource-efficient Autonomous Training Algorithms for Deep Learning"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Overview<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Training deep-learning models requires large amounts of computing power, time, and energy. It also often depends on carefully chosen <strong>hyperparameters<\/strong>, such as the learning rate, which determine how a neural network learns.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Led by Frank Schneider at the University of T\u00fcbingen, this project developed more autonomous and resource-efficient training methods. The goal was to enable training algorithms to adjust important hyperparameters automatically, reducing the need for manual tuning. The project ran from 2022 to 2024.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Key results<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Developed training methods that can adapt hyperparameters during the training process.<\/li>\n\n\n\n<li>Investigated techniques for reducing the computational demands and energy consumption of deep-learning training.<\/li>\n\n\n\n<li>Advanced second-order and curvature-based methods, which use information about the shape of the optimization problem to improve training.<\/li>\n\n\n\n<li>Established <strong>AlgoPerf: Training Algorithms<\/strong>, a community benchmark for reliably comparing how quickly different training algorithms reach a given level of model performance.<\/li>\n\n\n\n<li>Developed AlgoPerf through an open collaboration involving more than 20 researchers from academia and industry. The initiative was co-chaired by George Dahl of Google DeepMind and Frank Schneider of the University of T\u00fcbingen.<\/li>\n\n\n\n<li>Made the algorithms, benchmark results, and research findings publicly available so that academic and industrial laboratories can reproduce and build on them.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Why it matters<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Deep learning is often limited by the cost of training large neural networks. More autonomous training algorithms could reduce the need for expert intervention while lowering computational costs and energy consumption.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">AlgoPerf is particularly important because it provides a common evaluation standard: training methods can be compared according to how efficiently they achieve a specified level of performance, rather than only by their final results. This may accelerate progress toward more efficient and accessible deep learning.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">This project produced the following peer-reviewed publications:&nbsp;<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Tatzel, L., Hennig, P., &amp; Schneider, F. (2022). Late-Phase Second-Order Training. \u00a0NeurIPS 2022 Workshop: Has it Trained Yet?. [open access]<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Eschenhagen, R., Immer, A., Turner, R., Schneider, F., &amp; Hennig, P. (2023). Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures. Advances in Neural Information Processing Systems 36 (NeurIPS). [open access]<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Kasimbeg, P., Schneider, F., Eschenhagen, R., Bae, J. et al. (2025). Accelerating neural network training: An analysis of the AlgoPerf competition. International Conference on Learning Representations (ICLR). [open access]<\/li>\n\n\n\n<li>Fernandez, A., Schneider, F., Mahsereci, M., &amp; Hennig, P. (2025) Connecting Parameter Magnitudes and Hessian Eigenspaces at Scale using Sketched Methods. Transactions on Machine Learning Research (TMLR). [open access]<\/li>\n<\/ul>\n","protected":false},"featured_media":0,"parent":0,"template":"","meta":{"_acf_changed":true},"cv-tag":[290],"user-group":[],"class_list":["post-12048","researchfund-project","type-researchfund-project","status-publish","hentry","cv-tag-university-of-tubingen"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.7 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Resource-efficient Autonomous Training Algorithms for Deep Learning - Cyber Valley<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cyber-valley.de\/de\/researchfund-project\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Resource-efficient Autonomous Training Algorithms for Deep Learning - Cyber Valley\" \/>\n<meta property=\"og:description\" content=\"Overview Training deep-learning models requires large amounts of computing power, time, and energy. It also often depends on carefully chosen hyperparameters, such as the learning rate, which determine how a neural network learns. Led by Frank Schneider at the University of T\u00fcbingen, this project developed more autonomous and resource-efficient training methods. The goal was to [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cyber-valley.de\/de\/researchfund-project\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\/\" \/>\n<meta property=\"og:site_name\" content=\"Cyber Valley\" \/>\n<meta property=\"article:modified_time\" content=\"2026-09-01T12:10:16+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data1\" content=\"2\u00a0Minuten\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/researchfund-project\\\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\\\/\",\"url\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/researchfund-project\\\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\\\/\",\"name\":\"Resource-efficient Autonomous Training Algorithms for Deep Learning - Cyber Valley\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/#website\"},\"datePublished\":\"2026-08-19T14:20:57+00:00\",\"dateModified\":\"2026-09-01T12:10:16+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/researchfund-project\\\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/cyber-valley.de\\\/de\\\/researchfund-project\\\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/researchfund-project\\\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Resource-efficient Autonomous Training Algorithms for Deep Learning\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/#website\",\"url\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/\",\"name\":\"Cyber Valley\",\"description\":\"From lab to market\",\"publisher\":{\"@id\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/#organization\",\"name\":\"Cyber Valley GmbH\",\"url\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/cyber-valley.de\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/CV_Logo-Lockup_RGB_20210412.svg\",\"contentUrl\":\"https:\\\/\\\/cyber-valley.de\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/CV_Logo-Lockup_RGB_20210412.svg\",\"width\":1112,\"height\":317,\"caption\":\"Cyber Valley GmbH\"},\"image\":{\"@id\":\"https:\\\/\\\/cyber-valley.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.linkedin.com\\\/company\\\/cybervalley\\\/\",\"https:\\\/\\\/www.instagram.com\\\/cybervalley_bw\\\/\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UCPi8donVDkcOeM4C0CfP5fg\",\"https:\\\/\\\/bsky.app\\\/profile\\\/cyber-valley.bsky.social\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Resource-efficient Autonomous Training Algorithms for Deep Learning - Cyber Valley","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/cyber-valley.de\/de\/researchfund-project\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\/","og_locale":"de_DE","og_type":"article","og_title":"Resource-efficient Autonomous Training Algorithms for Deep Learning - Cyber Valley","og_description":"Overview Training deep-learning models requires large amounts of computing power, time, and energy. It also often depends on carefully chosen hyperparameters, such as the learning rate, which determine how a neural network learns. Led by Frank Schneider at the University of T\u00fcbingen, this project developed more autonomous and resource-efficient training methods. The goal was to [&hellip;]","og_url":"https:\/\/cyber-valley.de\/de\/researchfund-project\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\/","og_site_name":"Cyber Valley","article_modified_time":"2026-09-01T12:10:16+00:00","twitter_card":"summary_large_image","twitter_misc":{"Gesch\u00e4tzte Lesezeit":"2\u00a0Minuten"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/cyber-valley.de\/de\/researchfund-project\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\/","url":"https:\/\/cyber-valley.de\/de\/researchfund-project\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\/","name":"Resource-efficient Autonomous Training Algorithms for Deep Learning - Cyber Valley","isPartOf":{"@id":"https:\/\/cyber-valley.de\/de\/#website"},"datePublished":"2026-08-19T14:20:57+00:00","dateModified":"2026-09-01T12:10:16+00:00","breadcrumb":{"@id":"https:\/\/cyber-valley.de\/de\/researchfund-project\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/cyber-valley.de\/de\/researchfund-project\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/cyber-valley.de\/de\/researchfund-project\/x_resource-efficient-autonomous-training-algorithms-for-deep-learning\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/cyber-valley.de\/de\/"},{"@type":"ListItem","position":2,"name":"Resource-efficient Autonomous Training Algorithms for Deep Learning"}]},{"@type":"WebSite","@id":"https:\/\/cyber-valley.de\/de\/#website","url":"https:\/\/cyber-valley.de\/de\/","name":"Cyber Valley","description":"From lab to market","publisher":{"@id":"https:\/\/cyber-valley.de\/de\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/cyber-valley.de\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/cyber-valley.de\/de\/#organization","name":"Cyber Valley GmbH","url":"https:\/\/cyber-valley.de\/de\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/cyber-valley.de\/de\/#\/schema\/logo\/image\/","url":"https:\/\/cyber-valley.de\/wp-content\/uploads\/2026\/05\/CV_Logo-Lockup_RGB_20210412.svg","contentUrl":"https:\/\/cyber-valley.de\/wp-content\/uploads\/2026\/05\/CV_Logo-Lockup_RGB_20210412.svg","width":1112,"height":317,"caption":"Cyber Valley GmbH"},"image":{"@id":"https:\/\/cyber-valley.de\/de\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.linkedin.com\/company\/cybervalley\/","https:\/\/www.instagram.com\/cybervalley_bw\/","https:\/\/www.youtube.com\/channel\/UCPi8donVDkcOeM4C0CfP5fg","https:\/\/bsky.app\/profile\/cyber-valley.bsky.social"]}]}},"_links":{"self":[{"href":"https:\/\/cyber-valley.de\/de\/wp-json\/wp\/v2\/researchfund-project\/12048","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cyber-valley.de\/de\/wp-json\/wp\/v2\/researchfund-project"}],"about":[{"href":"https:\/\/cyber-valley.de\/de\/wp-json\/wp\/v2\/types\/researchfund-project"}],"wp:attachment":[{"href":"https:\/\/cyber-valley.de\/de\/wp-json\/wp\/v2\/media?parent=12048"}],"wp:term":[{"taxonomy":"cv-tag","embeddable":true,"href":"https:\/\/cyber-valley.de\/de\/wp-json\/wp\/v2\/cv-tag?post=12048"},{"taxonomy":"user-group","embeddable":true,"href":"https:\/\/cyber-valley.de\/de\/wp-json\/wp\/v2\/user-group?post=12048"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}