- Delete `BOM Update Batch` table on 'Completed' log, to save space
- Hide Progress section on 'Completed' log
- Enqueue `on_submit` for 'Update Cost' job, getting leaf boms could take time for huge DBs. Users have to wait for screen to unfreeze.
- Add error handling to `process_boms_cost_level_wise` (Called via cron job and on submit, both in background)
* refactor: DB independent quoting and truthy/falsy values
* style: reformat to black spec
* fix: ifnull -> coalesce
* fix: coalesce -> Coalesce
* fix: revert pypika comparison
* refactor: convert queries to QB
* fix: incorrect value types for query
`=` query makes no sense with list of values
* fix: remove warehouse docstatus condition
* fix: keep using base rate as rate
Co-authored-by: Ankush Menat <ankush@frappe.io>
fix(UX): use doc.status for JC status
- Use doc.status directly for indicator - single source of truth
- Update status to cancelled when doc is cancelled
- Remove `auto_commit_on_many_writes` in `update_cost_in_level()` as commits happen every N BOMs
- Auto commit every 50 BOMs
- test: Remove hacky `frappe.flags.in_test` returns
- test: Enqueue `now` if in tests (for update cost and replace bom)
- Replace BOM: Copy bom object to `_doc_before_save` so that version.py finds a difference between the two
- Replace BOM: Add reference to version
- Update Cost: Unset `processed_boms` if Log is completed (useless after completion)
- test: `update_cost_in_all_boms_in_test` works close to actual prod implementation (only call Cron job manually)
- Test: use `enqueue_replace_bom` so that test works closest to production behaviour
Co-authored-by: Ankush Menat <ankushmenat@gmail.com>
- Utility to update cost in all BOMs without cron jobs or background jobs (run immediately)
- Re-use util wherever all bom costs are to be updated
- Skip explicit commits if in test
- Specify company in test records (dirty data sometimes, company wh mismatch)
- Skip background jobs queueing if in test
- `get_valuation_rate`: if no bins are found return 0, SLEs do not exist either
- `get_valuation_rate`: Compute average valuation rate via query
- `get_rm_rate_map`: set order_by as None to avoid creating sort index (modified) each time query runs (seen in process list)
- BOM Update Batch: add status field and hide `boms_updated` so that users can see progress without loading all updated boms (too much data)
- BOM Update Batch: set batch row status to completed after job runs
- BOM Update Log: remove `parent_boms` field (just pass parent boms to processing function) & remove Paused state (not used)
- Move job to long queue to avoid choking default queue
- `update_cost_in_boms`: use `get_doc` as each BOM is accessed only once. Use `for_update` to lock BOM row
- Commit after every 100 BOMs
- This was done due to stale reads while the background jobs tried updating status of the log
- Added a table where all bom jobs within log will be tracked with what level they are processing
- Cron job will check if table jobs are all processed every 5 mins
- If yes, it will prepare parents and call `process_boms_cost_level_wise` to start next level
- If pending jobs, do nothing
- Current BOM Level is being tracked that helps adding rows to the table
- Individual bom cost jobs (that are queued) will process and update boms > will update BOM Update Batch table row with list of updated BOMs
- If `Update Cost` job is ongoing, then block creation of new ones since all BOMs are updated
- `db_update` in `calculate_rm_cost` only if changed values to reduce redundant row updates
- Misc: Use variable for batch size
- Separate getting dependants and checking if they are valid (loop within loop led to redundant processing that slowed down function)
- Adding to above, the same dependant(parent) was repeatedly processed as many children shared it. Expensive.
- Use a parent-child map similar to child-parent map to check if all children are resolved
- `map.get()` reduced time: 10 mins -> 0.9s~1 second (as compared to `get_cached_doc` or query)
- Total time: 17 seconds to process 6599 leaf boms and 4.2L parent boms
- Previous Total time: >10 mins (I terminated it due to not wanting to waste time XD)
* fix: date filter
fix from date to to date filter btw those days
* fix: remove unnecessary conditions
Co-authored-by: Ankush Menat <ankushmenat@gmail.com>
- Generate RM-Rate map from Items table (will include subassembly items with rate)
- Function to reset exploded item rate from above map
- `db_update` exploded item rate only if rate is changed
- Via Update Cost, only update exploded items rate, do not regenerate table again
- Exploded Items are regenerated on Save and Replace BOM job
- `calculate_exploded_cost` is run only via non doc events (Update Cost button, Update BOMs Cost Job)
- Process BOMs level wise and Pause after level is complete
- Cron job will resume Paused jobs, which will again process the new level and pause at the end
- This will go on until all BOMs are updated
- Added Progress section with fields to track updated BOMs in Log
- Cleanup: Add BOM Updation utils file to contain helper functions/sub-functions
- Cleanup: BOM Update Log file will only contain functions that are in direct context of the Log
Co-authored-by: Gavin D'souza <gavin18d@gmail.com>
- Use the right price list and currency to avoid rate conversion (1000/62.9), since rates are reset correctly now
- Use RM rate based on Price List in BOM. Non stock item has no valuation
- Doc is only used to iterate over items(which wont change) and change rate/amount of rows
- These changes are inserted in db via `db_update`, so no harm
- Tested locally: refetching cached doc after db update, reflects fresh data.
- Move `get_boms_in_bottom_up_order` in bom update tool’s file
- Remove repeated rm cost update from `update_cost`. `calculate_cost` handles RM cost update
- db_update children in `calculate_cost` optionally
- Don’t call `update_exploded_items` and regenerate exploded items in `update_cost`. They will stay the same (except cost)
- Create child-parent map once and fetch value from child key to get parents
- Get parents recursively for a leaf node (get all ancestors)
- Approx. 44 secs for 4lakh 70k boms
- Block excess transfer of items if not allowed in settings
- Behaviour made consistent with js behaviour (button disappears if not pending and not allowed in settings)
- Test for same case
- `set_missing_values` in SE will set actual qty, transfer qty and calculate rate/amount
- Re-use `set_missing_values` wherever SE is doc is being mapped
* fix: Cannot start Job strat if related to Work Order not started yet
* fix: Cannot start Job strat if related to Work Order not started yet
* test
* test
* fix siders
* PR review
* chore: Code cleanup
- Better short circuit for if condition (make it such that both conditions dont always have to be computed)
- Remove `r.message` extraction by avoiding `then()`
* chore: Remove unnecessary json change
Co-authored-by: marination <maricadsouza221197@gmail.com>
- Check for pending qty in child items to show/hide "Start" button
- If no qty needed to transfer (FG qty is fulfilled), but RM qty pending: map pending in SE with For Quantity = 0
- Renamed public function`update_new_bom` to `update_new_bom_in_bom_items`
- Replaced `get_cached_doc` with `get_doc`
- Removed click progress bar (drive through update log)
- Removed `bom_obj.update_new_bom()`, was redundant. Did same job as `update_new_bom_in_bom_items`
- Removed `update_new_bom()` in `bom.py`, unused.
- Prettier query formatting
- `update_type` annotated as non optional Literal
- Removed redundant use of JobTimeoutException
- Corrected type annotations in `create_bom_update_log()`
Production Plan tables for po_items and sub_assembly_items are prepared
client side so both dont exist at time of first save or modifying and
hence any "links" created are invalid. This change retains temporary
name so it can be relinked server side after naming is performed.
Co-Authored-By: Marica <maricadsouza221197@gmail.com>
- Added Typing
- Moved all job business logic to bom update log
- Added `run_bom_job` that handles errors and runs either of two methods
- UX: Replace button disabled until both inputs are filled
- Show log creation message on UI for correctness
- APIs return log document as result
- Converted raw sql to QB
- Created BOM Update Log that will handle queued job status and failures
- Moved validation and BG job to thus new doctype
- BOM Update Tool only works as an endpoint
* fix: show status in job card list view in Draft mode
(cherry picked from commit fa32fc3c832196004154055ac1b2334d6202c261)
* fix: job card - sub operations table status misbehaviour on pause / resume Job Card
(cherry picked from commit 7b8723445ee695e88590bb833f60a0c4731106fa)
Co-authored-by: Anoop Kurungadam <anoop@earthianslive.com>
