A question about the storeSide of hashjoiner

sql
(jemy Lan) #1

(CockroachDB version 2.1.2)

When building a hashjoiner, if sql.distsql.temp_storage.workmem is not enough, the storeSide will be set to the side of AddRow() function was being executed at that moment, then jump to “hjConsumingStoredSide” state. The consumeStoredSide() function will fully consume the storeSide. The storeSide is probably left or right.

(pkg/sql/distsqlrun/hashjoiner.go:build())

So it doesn’t look like the comments in “hjConsumingStoredSide” that It will fully consume the right side. (maybe is left side)

0 Likes

(Alfonso Subiotto Marqués) #2

@xiaolanzao66 you’re correct that the comment is not correct, and there are cases where we will move to consumer the left side. However, note that we buffer at most hashJoinerInitialBufferSize in either row container (4MiB) so if that limit is hit before a memory limit in AddRow (by default 64MiB), we will always consume the right side:

		leftUsage := h.rows[leftSide].MemUsage()
		rightUsage := h.rows[rightSide].MemUsage()

		if leftUsage >= h.initialBufferSize && rightUsage >= h.initialBufferSize {
			// Both sides have reached the buffer size limit. Move on to storing and
			// fully consuming the right side.
			log.VEventf(h.Ctx, 1, "buffer phase found no short stream with buffer size %d", h.initialBufferSize)
			return setStoredSideTransition(rightSide)
		}

So in practice it will usually be the right side, unless workmem is less than 4MiB. Feel free to submit a PR to change the comment!

0 Likes

(jemy Lan) #3

When useTempStorage was true, h.initialBufferSize would be overrided to be half of this processor’s memory limit(64MiB/2=32MiB), and it was no longer 4MiB.

(pkg/sql/distsqlrun/hashjoiner.go:newHashJoiner())

if h.useTempStorage {
		// Limit the memory use by creating a child monitor with a hard limit.
		// The hashJoiner will overflow to disk if this limit is not enough.
		limit := h.flowCtx.testingKnobs.MemoryLimitBytes
		if limit <= 0 {
			limit = settingWorkMemBytes.Get(&st.SV)
		}
		limitedMon := mon.MakeMonitorInheritWithLimit("hashjoiner-limited", limit, flowCtx.EvalCtx.Mon)
		limitedMon.Start(ctx, flowCtx.EvalCtx.Mon, mon.BoundAccount{})
		h.MemMonitor = &limitedMon
		h.diskMonitor = NewMonitor(ctx, flowCtx.diskMonitor, "hashjoiner-disk")
		// Override initialBufferSize to be half of this processor's memory
		// limit. We consume up to h.initialBufferSize bytes from each input
		// stream.
		h.initialBufferSize = limit / 2
	} else {
		h.MemMonitor = NewMonitor(ctx, flowCtx.EvalCtx.Mon, "hashjoiner-mem")
	}

So it may nerver go to

if leftUsage >= h.initialBufferSize && rightUsage >= h.initialBufferSize {
			// Both sides have reached the buffer size limit. Move on to storing and
			// fully consuming the right side.
			log.VEventf(h.Ctx, 1, "buffer phase found no short stream with buffer size %d", h.initialBufferSize)
			return setStoredSideTransition(rightSide)
		}

I have try many times in different situations (with sql.distsql.temp_storage.joins = true), It always go to

if err := h.rows[side].AddRow(h.Ctx, row); err != nil {
			// If this error is a memory limit error, move to hjConsumingStoredSide.
			h.storedSide = side
			if spilled, spillErr := h.maybeSpillToDisk(err); spilled {
				addErr := h.storedRows.AddRow(h.Ctx, row)
				if addErr == nil {
					return hjConsumingStoredSide, nil, nil
				}

and the storeSide may be left or right.

0 Likes