Reading the documentation, I guessed at the potential, but without real development experience it stayed just a theory in my mind.<\/p>\n
Today I\u2019m working on\u00a0<\/em>a real time big data tool <\/em>that executes thousands of jobs every hour, so my knowledge of this architecture is much better than in the past.<\/p>\n In this article I\u2019m going to show you how I discovered queues and jobs, and what configurations helped me to treat a large amount of data in real time while keeping server resources cost-friendly.<\/p>\n When a PHP application receives an incoming http request, our code is executed sequentially, step by step,<\/strong> until the request\u2019s execution ends and a response is returned back to the client (e.g. the browser).<\/p>\n That synchronous behavior is really intuitive, predictable, and simple to understand.<\/p>\n I launch an http request to my endpoint, the application retrieves data from the database, converts it into an appropriate format, execute some additional task, and sends it back. It\u2019s linear.<\/p>\n Queues and jobs introduce asynchronous behaviors that break this linear flow. That\u2019s why I think these functions seemed a little strange to me at the beginning.<\/p>\n But sometimes a time-consuming task involves completing an execution cycle due to an incoming http request, e.g., sending an email notification to all team members of a project.<\/p>\n It could mean sending six or ten emails and it could take four or five seconds to be completed. So every time a user clicks on that button, they need to wait five seconds before they can continue using the app.<\/p>\n More the application grows, the worse this problem gets. Good question.<\/p>\n A Job is a class that implements the \u201chandle<\/em>\u201d method that will contains the logic we want to execute at the time the job is scheduled to be executed.<\/p>\n As mentioned above the main reason to encapsulate a piece of code into a Job <\/strong>is to execute a time consuming task without forcing the user to wait its execution.<\/p>\n This is a legitimate question.<\/p>\n Sending emails is the most common example used in articles that talk about queues, but I want to tell you what I needed to do in my real experience.<\/p>\n As product owner it\u2019s really important for me to keep users\u2019 journey information in sync with our marketing and customer support tools. So, based on user actions, we update user information to various external software via APIs (a.k.a. external http calls) for marketing and customer care purposes.<\/p>\n One of the most used endpoint in my application could sends 10 emails and execute 3 http call to external services to be completed. No user would wait all this time, much more likely they would stop using my application.<\/p>\n Thanks to queues, I can encapsulate all these tasks in dedicated classes, pass in the contructor the information needed to do their job, and schedule their execution for later in the background so my controller can return a response immediately.<\/p>\n I don\u2019t need to wait until all of these processes are completed before returning a response; rather, I\u2019ll wait only for the time needed to publish them in the queue.<\/p>\n This could mean the difference between 10 seconds and 10 milliseconds!!!<\/p>\n This is a classic \u201cpublisher\/consumer\u201d architecture. We\u2019ve just published our jobs in the queue from the controller, so now we are going to understand how the queue is consumed, and finally jobs executed.<\/p>\n To consume a queue we need to run one of the most popular artisan command:<\/p>\n As reported in the Laravel documentation<\/a>:<\/p>\n Great! Laravel provides a ready-to-use interface to put jobs in a queue and a ready-to-use command to pull jobs from the queue and execute their code in the background.<\/p>\n This was another \u201cstrange thing\u201d at the beginning. I think it\u2019s normal discovering new things. Also I have experienced this phase of study so I write these articles to help me organize my skills and at the same time I can help other developers to expand their knowledge.<\/p>\n If a job fails wen firing an exception, the queue: work<\/strong> command will stop its work.<\/p>\n To keep the queue: work<\/strong> process running permanently (consuming your queues), you should use a process monitor such as Supervisor<\/strong> to ensure that the queue: work<\/strong> command does not stop running even if a job fires an exception.<\/p>\n The supervisor restarts the command after it goes down, starting again from the next job, abandoning the one that failed.<\/p>\n Well.<\/p>\n Jobs will be executed in the background on your server, no longer depending on an HTTP request. This introduces some changes that I had to consider when implementing the job\u2019s code.<\/p>\n Here is the most important in my mind:<\/p>\n Running in background you can\u2019t see immediately if your job generate errors.<\/p>\n You will no longer have immediate feedback like running an http request from your browser. If the job fails he will do it silently, without anyone noticing.<\/p>\n We will come back to this problem later.<\/b><\/p>\n The http request is gone<\/em>. Your code will be executed from cli.<\/p>\n If you need request parameters to accomplish your tasks you need to pass them in the job\u2019s constructor to use later during execution:<\/p>\n The session is gone.<\/em>\u00a0In the same way, you won\u2019t know the identity of the user who was logged in, so if you need the user information to accomplish the task, you need to pass the user object to the job\u2019s constructor:<\/p>\n Unfortunately in many cases it isn\u2019t enought. Using a single queue and consumer it may soon become useless.<\/p>\n Queues are FIFO buffers (F<\/strong>irst\u00a0I<\/strong>n\u00a0F<\/strong>irst\u00a0O<\/strong>ut). If you schedule many jobs, also of different types, they need to wait for others to execute their scheduled tasks before being completed<\/p>\n There are two ways to scale. In this way, five jobs will be pulled from the queue at a time, speeding up the queue\u2019s consumption.<\/p>\n You could also create specific queues for each job \u201ctype\u201d you are launching, with a dedicated consumer for each queue.<\/p>\n In this way, each queue will be consumed independently without having to wait for the execution of the other types of jobs.<\/p>\n Laravel Horizon<\/strong><\/a>\u00a0is a queue manager<\/strong>\u00a0that gives you full control over how many queues you want to set up and the ability to organize consumers, allowing developers to put these two strategies together and implement one that fits your scalability needs.<\/p>\n It all starts by running\u00a0 In the example above Horizon will start three queues with three processes assigned to consume each queue.<\/p>\nA gentle introduction<\/h3>\n
\nAlso read:<\/span> What Is Walmart Credit Card Grace Period? Explained<\/a><\/span>\nWhat is a “Job”?<\/h3>\n
use Illuminate\\Contracts\\Queue\\ShouldQueue;\r\nuse Illuminate\\Foundation\\Bus\\Dispatchable;\r\nuse Illuminate\\Queue\\InteractsWithQueue;\r\nuse Illuminate\\Bus\\Queueable;\r\n\r\nclass CallExternalAPI implements ShouldQueue\r\n{\r\n use Dispatchable;\r\n use InteractsWithQueue;\r\n use Queueable;\r\n \r\n \/**\r\n * @var string\r\n *\/\r\n protected $url;\r\n\r\n \/**\r\n * Create a new job instance.\r\n *\r\n * @param array $Data\r\n *\/\r\n public function __construct($url)\r\n {\r\n $this->url = $url;\r\n }\r\n \r\n\r\n \/**\r\n * Execute what you want.\r\n *\r\n * @return void\r\n * @throws \\Throwable\r\n *\/\r\n public function handle()\r\n {\r\n file_get_contents($this->url);\r\n }\r\n}<\/pre>\nWhat do you mean with \u201ctime consuming tasks\u201d?<\/h3>\n
class ProjectController \r\n{\r\n public function store(Request $request)\r\n {\r\n $project = Project::create($request->all());\r\n \r\n \/\/ Defer NotifyMembers, TagUserActive, NotifyToProveSource \r\n \/\/ passing the information needed to do their job\r\n Notification::queue(new NotifyMembers($project->owners));\r\n $this->dispatch(new TagUserAsActive($project->owners));\r\n $this->dispatch(new NotifyToProveSource($project->owners));\r\n \r\n return $project;\r\n }\r\n}<\/pre>\nWho executes these jobs after posting them in the queue?<\/h3>\n
![\"Laravel](\"https:\/\/www.inspector.dev\/wp-content\/uploads\/2019\/09\/1_Evvsj-ibqWBiAemzFMHzdA.png\")
<\/p>\nphp artisan queue:work<\/pre>\n
The role of Supervisor<\/h3>\n
How can I be aware if the job code fails?<\/h3>\n
You don\u2019t have the HTTP request<\/h3>\n
\/\/ A job class example\r\nclass TagUserJob implements ShouldQueue\r\n{\r\n public $data;\r\n \r\n public function __construct(array $data)\r\n {\r\n $this->data = $data;\r\n }\r\n}\r\n\r\n\/\/ Put the job in the queue from your controller\r\n$this->dispatch(new TagUserJob($request->all()));<\/pre>\nYou don\u2019t know who the logged user is<\/h3>\n
\/\/ A job class example\r\nclass TagUserJob implements ShouldQueue\r\n{\r\n public $user;\r\n \r\n public function __construct(User $user)\r\n {\r\n $this->user= $user;\r\n }\r\n}\r\n\r\n\/\/ Put the job in the queue from your controller\r\n$this->dispatch(new TagUserJob($request->user()));<\/pre>\nUnderstand how to scale<\/h3>\n
\nAlso read:<\/span> 10 Best AI Music Generator In 2025 (Royalty Free Music Generation)<\/a><\/span>\nMultiple consumers for a queue<\/h3>\n
![\"Multiple](\"https:\/\/www.inspector.dev\/wp-content\/uploads\/2019\/09\/1_B95oVhCZoVkv_2jhV8nHew.png\")
<\/p>\nSingle-purpose queues<\/h3>\n
![\"Single-purpose](\"https:\/\/www.inspector.dev\/wp-content\/uploads\/2019\/09\/1_1ICT3D0C0bie7Xg-wrk7gg.png\")
<\/p>\nTowards Horizon<\/h3>\n
php artisan horizon<\/code>\u00a0instead\u00a0php artisan queue:work<\/code>. This command scans your\u00a0horizon.php<\/code>\u00a0configuration file & starts a number of queue workers based on the configuration:<\/p>\n'production' => [\r\n 'supervisor-1' => [\r\n 'connection' => \"redis\",\r\n 'queue' => ['adveritisement', 'logs', 'phones'],\r\n 'processes' => 9,\r\n 'tries' => 3,\r\n 'balance' => 'simple', \/\/ could be simple, auto, or null\r\n ]\r\n]<\/pre>\n
![\"laravel](\"https:\/\/www.inspector.dev\/wp-content\/uploads\/2019\/09\/1_XHkernGbcjpSSjNID81Mdw.png\")
<\/p>\n