- Implement Log clearing interface in BOM Update Log
- Add additional info in sidebar: Log clearing only happens for 'Update Cost' type logs
- 'Replace BOM' logs have important info and is used in BOM timeline, so we'll let users decide if they wanna keep or discard it
- Delete `BOM Update Batch` table on 'Completed' log, to save space
- Hide Progress section on 'Completed' log
- Enqueue `on_submit` for 'Update Cost' job, getting leaf boms could take time for huge DBs. Users have to wait for screen to unfreeze.
- Add error handling to `process_boms_cost_level_wise` (Called via cron job and on submit, both in background)
* refactor: DB independent quoting and truthy/falsy values
* style: reformat to black spec
* fix: ifnull -> coalesce
* fix: coalesce -> Coalesce
* fix: revert pypika comparison
* refactor: convert queries to QB
* fix: incorrect value types for query
`=` query makes no sense with list of values
* fix: remove warehouse docstatus condition
* fix: keep using base rate as rate
Co-authored-by: Ankush Menat <ankush@frappe.io>
fix(UX): use doc.status for JC status
- Use doc.status directly for indicator - single source of truth
- Update status to cancelled when doc is cancelled
- Remove `auto_commit_on_many_writes` in `update_cost_in_level()` as commits happen every N BOMs
- Auto commit every 50 BOMs
- test: Remove hacky `frappe.flags.in_test` returns
- test: Enqueue `now` if in tests (for update cost and replace bom)
- Replace BOM: Copy bom object to `_doc_before_save` so that version.py finds a difference between the two
- Replace BOM: Add reference to version
- Update Cost: Unset `processed_boms` if Log is completed (useless after completion)
- test: `update_cost_in_all_boms_in_test` works close to actual prod implementation (only call Cron job manually)
- Test: use `enqueue_replace_bom` so that test works closest to production behaviour
Co-authored-by: Ankush Menat <ankushmenat@gmail.com>
- Utility to update cost in all BOMs without cron jobs or background jobs (run immediately)
- Re-use util wherever all bom costs are to be updated
- Skip explicit commits if in test
- Specify company in test records (dirty data sometimes, company wh mismatch)
- Skip background jobs queueing if in test
- `get_valuation_rate`: if no bins are found return 0, SLEs do not exist either
- `get_valuation_rate`: Compute average valuation rate via query
- `get_rm_rate_map`: set order_by as None to avoid creating sort index (modified) each time query runs (seen in process list)
- BOM Update Batch: add status field and hide `boms_updated` so that users can see progress without loading all updated boms (too much data)
- BOM Update Batch: set batch row status to completed after job runs
- BOM Update Log: remove `parent_boms` field (just pass parent boms to processing function) & remove Paused state (not used)
- Move job to long queue to avoid choking default queue
- `update_cost_in_boms`: use `get_doc` as each BOM is accessed only once. Use `for_update` to lock BOM row
- Commit after every 100 BOMs
- This was done due to stale reads while the background jobs tried updating status of the log
- Added a table where all bom jobs within log will be tracked with what level they are processing
- Cron job will check if table jobs are all processed every 5 mins
- If yes, it will prepare parents and call `process_boms_cost_level_wise` to start next level
- If pending jobs, do nothing
- Current BOM Level is being tracked that helps adding rows to the table
- Individual bom cost jobs (that are queued) will process and update boms > will update BOM Update Batch table row with list of updated BOMs
- If `Update Cost` job is ongoing, then block creation of new ones since all BOMs are updated
- `db_update` in `calculate_rm_cost` only if changed values to reduce redundant row updates
- Misc: Use variable for batch size
- Separate getting dependants and checking if they are valid (loop within loop led to redundant processing that slowed down function)
- Adding to above, the same dependant(parent) was repeatedly processed as many children shared it. Expensive.
- Use a parent-child map similar to child-parent map to check if all children are resolved
- `map.get()` reduced time: 10 mins -> 0.9s~1 second (as compared to `get_cached_doc` or query)
- Total time: 17 seconds to process 6599 leaf boms and 4.2L parent boms
- Previous Total time: >10 mins (I terminated it due to not wanting to waste time XD)
* fix: date filter
fix from date to to date filter btw those days
* fix: remove unnecessary conditions
Co-authored-by: Ankush Menat <ankushmenat@gmail.com>
- Generate RM-Rate map from Items table (will include subassembly items with rate)
- Function to reset exploded item rate from above map
- `db_update` exploded item rate only if rate is changed
- Via Update Cost, only update exploded items rate, do not regenerate table again
- Exploded Items are regenerated on Save and Replace BOM job
- `calculate_exploded_cost` is run only via non doc events (Update Cost button, Update BOMs Cost Job)
- Process BOMs level wise and Pause after level is complete
- Cron job will resume Paused jobs, which will again process the new level and pause at the end
- This will go on until all BOMs are updated
- Added Progress section with fields to track updated BOMs in Log
- Cleanup: Add BOM Updation utils file to contain helper functions/sub-functions
- Cleanup: BOM Update Log file will only contain functions that are in direct context of the Log
Co-authored-by: Gavin D'souza <gavin18d@gmail.com>
- Use the right price list and currency to avoid rate conversion (1000/62.9), since rates are reset correctly now
- Use RM rate based on Price List in BOM. Non stock item has no valuation
- Doc is only used to iterate over items(which wont change) and change rate/amount of rows
- These changes are inserted in db via `db_update`, so no harm
- Tested locally: refetching cached doc after db update, reflects fresh data.
