Skip to content

3.4.3: prevent DB_Driver_WPDB::insert_record() from inserting empty meta values #1080

New issue
New issue
Closed
#1307
#1312
Closed
3.4.3: prevent DB_Driver_WPDB::insert_record() from inserting empty meta values#1080
#1307
#1312
Labels
enhancementperformance
@lkraav

Description

@lkraav
lkraav
opened on Apr 16, 2020

My platform Stream database is growing large, wp_stream_meta contains 63M rows.

Bug Report

Analysis with

SELECT count(*) AS `count` FROM `wp_stream_meta` WHERE `wp_stream_meta`.`meta_value` = ''

shows about 20-25% are rows with empty meta_value, which to me looks like pure database bloat.

Expected Behavior

wp_stream_meta should only contain rows with non-empty meta_value.

On read, any such parameters can simply be displayed empty in code.

Actual Behavior

Noticeable amount of wp_stream_meta table is rows with empty meta_value.

Example data log:

[22-Apr-2020 11:27:04 UTC] insert_meta: empty value, $key user_meta                                                                                                                            
[22-Apr-2020 11:27:05 UTC] insert_meta: empty value, $key user_name                                                                                                                            
[22-Apr-2020 11:27:05 UTC] insert_meta: empty value, $key user_id                                                                                                                              
[22-Apr-2020 11:27:05 UTC] insert_meta: empty value, $key user_meta                                                                                                                            
[22-Apr-2020 11:27:05 UTC] insert_meta: empty value, $key user_name                                                                                                                            
[22-Apr-2020 11:27:05 UTC] insert_meta: empty value, $key user_id
[22-Apr-2020 11:27:05 UTC] insert_meta: empty value, $key user_meta                                                                                                                            
[22-Apr-2020 11:27:06 UTC] insert_meta: empty value, $key user_meta                                                                                                                            
[22-Apr-2020 11:27:08 UTC] insert_meta: empty value, $key user_name                                                                                                                            
[22-Apr-2020 11:27:08 UTC] insert_meta: empty value, $key user_id
[22-Apr-2020 11:27:08 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:08 UTC] insert_meta: empty value, $key user_name
[22-Apr-2020 11:27:08 UTC] insert_meta: empty value, $key user_id
[22-Apr-2020 11:27:08 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:09 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:09 UTC] insert_meta: empty value, $key user_name
[22-Apr-2020 11:27:09 UTC] insert_meta: empty value, $key is_spam
[22-Apr-2020 11:27:09 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:09 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:09 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:09 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:28 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:29 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:29 UTC] insert_meta: empty value, $key user_name
[22-Apr-2020 11:27:29 UTC] insert_meta: empty value, $key user_id
[22-Apr-2020 11:27:29 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:29 UTC] insert_meta: empty value, $key user_name
[22-Apr-2020 11:27:29 UTC] insert_meta: empty value, $key user_id
[22-Apr-2020 11:27:29 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:30 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:31 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:31 UTC] insert_meta: empty value, $key user_name
[22-Apr-2020 11:27:31 UTC] insert_meta: empty value, $key is_spam
[22-Apr-2020 11:27:31 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:31 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:31 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:31 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:39 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:41 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:41 UTC] insert_meta: empty value, $key user_name
[22-Apr-2020 11:27:41 UTC] insert_meta: empty value, $key user_id
[22-Apr-2020 11:27:41 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:42 UTC] insert_meta: empty value, $key user_name
[22-Apr-2020 11:27:42 UTC] insert_meta: empty value, $key user_id
[22-Apr-2020 11:27:42 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:42 UTC] insert_meta: empty value, $key user_name
[22-Apr-2020 11:27:42 UTC] insert_meta: empty value, $key user_id
[22-Apr-2020 11:27:42 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:44 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:44 UTC] insert_meta: empty value, $key user_name
[22-Apr-2020 11:27:44 UTC] insert_meta: empty value, $key is_spam
[22-Apr-2020 11:27:44 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:44 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:44 UTC] insert_meta: empty value, $key user_meta
[22-Apr-2020 11:27:44 UTC] insert_meta: empty value, $key user_meta

Empty meta value connector analysis:

MariaDB> SELECT `wp_stream`.`connector` AS `connector`, count(*) AS `count` FROM `wp_stream_meta` LEFT JOIN `wp_stream` `wp_stream` ON `wp_stream_meta`.`record_id` = `wp_st
ream`.`ID` WHERE `wp_stream_meta`.`meta_value` = '' GROUP BY `wp_stream`.`connector` ORDER BY `count` DESC;                                                                                    
+-----------------------------------+---------+
| connector                         | count   |
+-----------------------------------+---------+
| comments                          | 9474347 |
| posts-viewed                      | 3549141 |
| settings                          |  437761 |
| users                             |  421972 |
| posts                             |  180445 |
| gravityforms                      |  151214 |
| media                             |    6104 |
| wordpressseo                      |    5229 |
| userswitching                     |    4057 |
| woocommerce                       |    2979 |
| installer                         |    1219 |
| taxonomies                        |     938 |
| menus                             |     455 |
| widgets                           |     402 |
| woocommerce-memberships-for-teams |     175 |
| NULL                              |      98 |
| buddypress                        |      38 |
| slack-memberships-api             |       4 |
+-----------------------------------+---------+
18 rows in set (5 min 35.921 sec)

Steps to Reproduce the Problem

  1. Run query at the top of opening comment.

System Information

  • 3.4.3
  • 5.4
  • 7.3

Solution proposal

Perhaps this code block

stream/classes/class-db-driver-wpdb.php

Lines 69 to 74 in 33dcfa8

// Insert record meta
foreach ( (array) $meta as $key => $vals ) {
foreach ( (array) $vals as $val ) {
$this->insert_meta( $record_id, $key, $val );
}
}

could have a simple if ( $val ) check before insert_meta() call?

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementperformance

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions