TypeError: PhpParser\\Lexer::getNextToken()

Created on 25 December 2022, almost 2 years ago
Updated 22 January 2024, 10 months ago

Hello,
during scan in upgrade status i get this error:

TypeError: PhpParser\\Lexer::getNextToken(): Return value must be of type int, null returned in /var/www/html/drupal9/vendor/nikic/php-parser/lib/PhpParser/Lexer.php on line 364 #0 /var/www/html/drupal9/vendor/nikic/php-parser/lib/PhpParser/ParserAbstract.php(209): PhpParser\\Lexer->getNextToken('&', Array, Array)\n#1 /var/www/html/drupal9/vendor/nikic/php-parser/lib/PhpParser/ParserAbstract.php(159): PhpParser\\ParserAbstract->doParse()\n#2 /var/www/html/drupal9/vendor/nikic/php-parser/lib/PhpParser/Parser/Multiple.php(51): PhpParser\\ParserAbstract->parse('<?php\\n\\n/**\\n * @...', Object(PhpParser\\ErrorHandler\\Throwing))\n#3 /var/www/html/drupal9/vendor/nikic/php-parser/lib/PhpParser/Parser/Multiple.php(32): PhpParser\\Parser\\Multiple->tryParse(Object(PhpParser\\Parser\\Php7), Object(PhpParser\\ErrorHandler\\Throwing), '<?php\\n\\n/**\\n * @...')\n#4 /var/www/html/drupal9/modules/contrib/upgrade_status/src/ThemeFunctionDeprecationAnalyzer.php(114): PhpParser\\Parser\\Multiple->parse('<?php\\n\\n/**\\n * @...')\n#5 /var/www/html/drupal9/modules/contrib/upgrade_status/src/ThemeFunctionDeprecationAnalyzer.php(58): Drupal\\upgrade_status\\ThemeFunctionDeprecationAnalyzer->analyzeFunction('ckeditor_theme', Object(Drupal\\Core\\Extension\\Extension))\n#6 /var/www/html/drupal9/modules/contrib/upgrade_status/src/DeprecationAnalyzer.php(413): Drupal\\upgrade_status\\ThemeFunctionDeprecationAnalyzer->analyze(Object(Drupal\\Core\\Extension\\Extension))\n#7 /var/www/html/drupal9/modules/contrib/upgrade_status/src/Form/UpgradeStatusForm.php(1349): Drupal\\upgrade_status\\DeprecationAnalyzer->analyze(Object(Drupal\\Core\\Extension\\Extension))\n#8 /var/www/html/drupal9/core/includes/batch.inc(295): Drupal\\upgrade_status\\Form\\UpgradeStatusForm::parseProject(Object(Drupal\\Core\\Extension\\Extension), false, Array)\n#9 /var/www/html/drupal9/core/includes/batch.inc(137): _batch_process()\n#10 /var/www/html/drupal9/core/includes/batch.inc(93): _batch_do()\n#11 /var/www/html/drupal9/core/modules/system/src/Controller/BatchController.php(55): _batch_page(Object(Symfony\\Component\\HttpFoundation\\Request))\n#12 [internal function]: Drupal\\system\\Controller\\BatchController->batchPage(Object(Symfony\\Component\\HttpFoundation\\Request))\n#13 /var/www/html/drupal9/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array)\n#14 /var/www/html/drupal9/core/lib/Drupal/Core/Render/Renderer.php(580): Drupal\\Core\\EventSubscriber\\EarlyRenderingControllerWrapperSubscriber->Drupal\\Core\\EventSubscriber\\{closure}()\n#15 /var/www/html/drupal9/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\\Core\\Render\\Renderer->executeInRenderContext(Object(Drupal\\Core\\Render\\RenderContext), Object(Closure))\n#16 /var/www/html/drupal9/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\\Core\\EventSubscriber\\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array)\n#17 /var/www/html/drupal9/vendor/symfony/http-kernel/HttpKernel.php(169): Drupal\\Core\\EventSubscriber\\EarlyRenderingControllerWrapperSubscriber->Drupal\\Core\\EventSubscriber\\{closure}()\n#18 /var/www/html/drupal9/vendor/symfony/http-kernel/HttpKernel.php(81): Symfony\\Component\\HttpKernel\\HttpKernel->handleRaw(Object(Symfony\\Component\\HttpFoundation\\Request), 1)\n#19 /var/www/html/drupal9/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\\Component\\HttpKernel\\HttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#20 /var/www/html/drupal9/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\\Core\\StackMiddleware\\Session->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#21 /var/www/html/drupal9/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\\Core\\StackMiddleware\\KernelPreHandle->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#22 /var/www/html/drupal9/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\\page_cache\\StackMiddleware\\PageCache->pass(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#23 /var/www/html/drupal9/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\\page_cache\\StackMiddleware\\PageCache->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#24 /var/www/html/drupal9/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\\Core\\StackMiddleware\\ReverseProxyMiddleware->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#25 /var/www/html/drupal9/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\\Core\\StackMiddleware\\NegotiationMiddleware->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#26 /var/www/html/drupal9/core/lib/Drupal/Core/DrupalKernel.php(707): Stack\\StackedHttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#27 /var/www/html/drupal9/index.php(19): Drupal\\Core\\DrupalKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request))\n#28 {main}, referer: https://xxxx/batch?id=73&op=start
🐛 Bug report
Status

Fixed

Version

4.0

Component

Code

Created by

🇵🇱Poland lordzik

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇵🇱Poland lordzik

    I'm still getting this. It happens on any project.

     drush upgrade_status:analyze google_cse
    
     [notice] Processing /var/www/html/drupal8/modules/contrib/google_cse.
     [warning] Undefined array key 403 Lexer.php:323
     [error]  TypeError: PhpParser\Lexer::getNextToken(): Return value must be of type int, null returned in PhpParser\Lexer->getNextToken() (line 364 of /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/Lexer.php) #0 /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/ParserAbstract.php(209): PhpParser\Lexer->getNextToken('&', Array, Array)
    #1 /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/ParserAbstract.php(159): PhpParser\ParserAbstract->doParse()
    #2 /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/Parser/Multiple.php(51): PhpParser\ParserAbstract->parse('<?php\n\n/**\n * @...', Object(PhpParser\ErrorHandler\Throwing))
    #3 /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/Parser/Multiple.php(32): PhpParser\Parser\Multiple->tryParse(Object(PhpParser\Parser\Php7), Object(PhpParser\ErrorHandler\Throwing), '<?php\n\n/**\n * @...')
    #4 /var/www/html/drupal8/modules/contrib/upgrade_status/src/ThemeFunctionDeprecationAnalyzer.php(114): PhpParser\Parser\Multiple->parse('<?php\n\n/**\n * @...')
    #5 /var/www/html/drupal8/modules/contrib/upgrade_status/src/ThemeFunctionDeprecationAnalyzer.php(58): Drupal\upgrade_status\ThemeFunctionDeprecationAnalyzer->analyzeFunction('google_cse_them...', Object(Drupal\Core\Extension\Extension))
    #6 /var/www/html/drupal8/modules/contrib/upgrade_status/src/DeprecationAnalyzer.php(420): Drupal\upgrade_status\ThemeFunctionDeprecationAnalyzer->analyze(Object(Drupal\Core\Extension\Extension))
    #7 /var/www/html/drupal8/modules/contrib/upgrade_status/src/Commands/UpgradeStatusCommands.php(264): Drupal\upgrade_status\DeprecationAnalyzer->analyze(Object(Drupal\Core\Extension\Extension), Array)
    #8 /var/www/html/drupal8/modules/contrib/upgrade_status/src/Commands/UpgradeStatusCommands.php(147): Drupal\upgrade_status\Commands\UpgradeStatusCommands->doAnalyze(Array, Array)
    #9 [internal function]: Drupal\upgrade_status\Commands\UpgradeStatusCommands->analyze(Array, Array)
    #10 /var/www/html/drupal8/vendor/consolidation/annotated-command/src/CommandProcessor.php(257): call_user_func_array(Array, Array)
    #11 /var/www/html/drupal8/vendor/consolidation/annotated-command/src/CommandProcessor.php(212): Consolidation\AnnotatedCommand\CommandProcessor->runCommandCallback(Array, Object(Consolidation\AnnotatedCommand\CommandData))
    #12 /var/www/html/drupal8/vendor/consolidation/annotated-command/src/CommandProcessor.php(176): Consolidation\AnnotatedCommand\CommandProcessor->validateRunAndAlter(Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
    #13 /var/www/html/drupal8/vendor/consolidation/annotated-command/src/AnnotatedCommand.php(311): Consolidation\AnnotatedCommand\CommandProcessor->process(Object(Symfony\Component\Console\Output\ConsoleOutput), Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
    #14 /var/www/html/drupal8/vendor/symfony/console/Command/Command.php(255): Consolidation\AnnotatedCommand\AnnotatedCommand->execute(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
    #15 /var/www/html/drupal8/vendor/symfony/console/Application.php(1039): Symfony\Component\Console\Command\Command->run(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
    #16 /var/www/html/drupal8/vendor/symfony/console/Application.php(275): Symfony\Component\Console\Application->doRunCommand(Object(Consolidation\AnnotatedCommand\AnnotatedCommand), Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
    #17 /var/www/html/drupal8/vendor/symfony/console/Application.php(149): Symfony\Component\Console\Application->doRun(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
    #18 /var/www/html/drupal8/vendor/drush/drush/src/Runtime/Runtime.php(118): Symfony\Component\Console\Application->run(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
    #19 /var/www/html/drupal8/vendor/drush/drush/src/Runtime/Runtime.php(48): Drush\Runtime\Runtime->doRun(Array, Object(Symfony\Component\Console\Output\ConsoleOutput))
    #20 /var/www/html/drupal8/vendor/drush/drush/drush.php(72): Drush\Runtime\Runtime->run(Array)
    #21 /var/www/html/drupal8/vendor/drush/drush/drush(4): require('/var/www/html/d...')
    #22 /var/www/html/drupal8/vendor/bin/drush(120): include('/var/www/html/d...')
    #23 {main}.
    TypeError: PhpParser\Lexer::getNextToken(): Return value must be of type int, null returned in /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/Lexer.php on line 364 #0 /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/ParserAbstract.php(209): PhpParser\Lexer->getNextToken('&', Array, Array)
    #1 /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/ParserAbstract.php(159): PhpParser\ParserAbstract->doParse()
    #2 /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/Parser/Multiple.php(51): PhpParser\ParserAbstract->parse('<?php\n\n/**\n * @...', Object(PhpParser\ErrorHandler\Throwing))
    #3 /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/Parser/Multiple.php(32): PhpParser\Parser\Multiple->tryParse(Object(PhpParser\Parser\Php7), Object(PhpParser\ErrorHandler\Throwing), '<?php\n\n/**\n * @...')
    #4 /var/www/html/drupal8/modules/contrib/upgrade_status/src/ThemeFunctionDeprecationAnalyzer.php(114): PhpParser\Parser\Multiple->parse('<?php\n\n/**\n * @...')
    #5 /var/www/html/drupal8/modules/contrib/upgrade_status/src/ThemeFunctionDeprecationAnalyzer.php(58): Drupal\upgrade_status\ThemeFunctionDeprecationAnalyzer->analyzeFunction('google_cse_them...', Object(Drupal\Core\Extension\Extension))
    #6 /var/www/html/drupal8/modules/contrib/upgrade_status/src/DeprecationAnalyzer.php(420): Drupal\upgrade_status\ThemeFunctionDeprecationAnalyzer->analyze(Object(Drupal\Core\Extension\Extension))
    #7 /var/www/html/drupal8/modules/contrib/upgrade_status/src/Commands/UpgradeStatusCommands.php(264): Drupal\upgrade_status\DeprecationAnalyzer->analyze(Object(Drupal\Core\Extension\Extension), Array)
    #8 /var/www/html/drupal8/modules/contrib/upgrade_status/src/Commands/UpgradeStatusCommands.php(147): Drupal\upgrade_status\Commands\UpgradeStatusCommands->doAnalyze(Array, Array)
    #9 [internal function]: Drupal\upgrade_status\Commands\UpgradeStatusCommands->analyze(Array, Array)
    #10 /var/www/html/drupal8/vendor/consolidation/annotated-command/src/CommandProcessor.php(257): call_user_func_array(Array, Array)
    #11 /var/www/html/drupal8/vendor/consolidation/annotated-command/src/CommandProcessor.php(212): Consolidation\AnnotatedCommand\CommandProcessor->runCommandCallback(Array, Object(Consolidation\AnnotatedCommand\CommandData))
    #12 /var/www/html/drupal8/vendor/consolidation/annotated-command/src/CommandProcessor.php(176): Consolidation\AnnotatedCommand\CommandProcessor->validateRunAndAlter(Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
    #13 /var/www/html/drupal8/vendor/consolidation/annotated-command/src/AnnotatedCommand.php(311): Consolidation\AnnotatedCommand\CommandProcessor->process(Object(Symfony\Component\Console\Output\ConsoleOutput), Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
    #14 /var/www/html/drupal8/vendor/symfony/console/Command/Command.php(255): Consolidation\AnnotatedCommand\AnnotatedCommand->execute(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
    #15 /var/www/html/drupal8/vendor/symfony/console/Application.php(1039): Symfony\Component\Console\Command\Command->run(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
    #16 /var/www/html/drupal8/vendor/symfony/console/Application.php(275): Symfony\Component\Console\Application->doRunCommand(Object(Consolidation\AnnotatedCommand\AnnotatedCommand), Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
    #17 /var/www/html/drupal8/vendor/symfony/console/Application.php(149): Symfony\Component\Console\Application->doRun(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
    #18 /var/www/html/drupal8/vendor/drush/drush/src/Runtime/Runtime.php(118): Symfony\Component\Console\Application->run(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
    #19 /var/www/html/drupal8/vendor/drush/drush/src/Runtime/Runtime.php(48): Drush\Runtime\Runtime->doRun(Array, Object(Symfony\Component\Console\Output\ConsoleOutput))
    #20 /var/www/html/drupal8/vendor/drush/drush/drush.php(72): Drush\Runtime\Runtime->run(Array)
    #21 /var/www/html/drupal8/vendor/drush/drush/drush(4): require('/var/www/html/d...')
    #22 /var/www/html/drupal8/vendor/bin/drush(120): include('/var/www/html/d...')
    #23 {main}
    TypeError: PhpParser\Lexer::getNextToken(): Return value must be of type int, null returned in PhpParser\Lexer->getNextToken() (line 364 of /var/www/html/drupal8/vendor/nikic/php-parser/lib/PhpParser/Lexer.php).
     [warning] Drush command terminated abnormally.
    

    Now i'm running: upgrade_status 4.0.0, Drupal 9.5.11 on PHP 8.1.24

  • Status changed to Active about 1 year ago
  • 🇨🇭Switzerland pjovanovic

    So after a long debugging session, I found the problem.
    The fix: update to the newest version, for me, it was v4.18.0
    The Problem: For some reason the version used is not the newest and the older version probably has a problem with php8.
    I found that the token processed in the Lexer.php file vendor/nikic/php-parser/lib/PhpParser/Lexer.php sometimes contains an "&" sign, which causes the parsing to fail. When I updated, the following if statement was added to address this problem:

     if ($token === '&')
    

    at Line 212. This should solve your issue and let you scan your modules.
    The package link: https://packagist.org/packages/nikic/php-parser#v4.18.0

  • Status changed to Fixed 10 months ago
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024